Access method, device and medium of a clustered file system

Abstract

This invention discloses a method, apparatus, and medium for accessing a clustered file system, applicable to virtualization and hyper-converged infrastructure. Based on block file data I / O operations, the Paxos algorithm is applied to determine the target node for lock request access. The entire process achieves node-to-node access without relying on the network. I / O operations are not limited by specific storage through read and write operations. An added lock region is used for inter-node competition for lock resources. Disk regions are partitioned to target metadata structures requiring concurrent access protection. Then, based on the Paxos algorithm and the kernel block layer interface, I / O requests are directly issued to allow target nodes to compete for lock resources. This avoids the problem of existing clustered file systems being unable to simultaneously achieve network independence and storage independence during concurrent access, improving the stability and usability of clustered files.

Description

Technical Field

[0001] This invention relates to the fields of virtualization and hyperconvergence, and in particular to a method, apparatus and medium for accessing a group file system. Background Technology

[0002] In the fields of virtualization and hyperconverged infrastructure, file systems are commonly used as the underlying technology for building storage pools, enabling them to fully leverage file system characteristics to provide virtualization system services such as snapshots, data backups, and expansion. Shared storage pools built on cluster robustness platforms can achieve advanced features such as high availability (HA), dynamic scheduling of virtual machine resources, and disaster recovery, forming the core capabilities of virtualization and hyperconverged systems. A core component of the cluster file system is the Distributed Lock Manager (DLM), which controls concurrent access to shared resources by nodes within the cluster.

[0003] Existing clustered file systems (DLMs) rely on either message-passing locking schemes (OpenDLM, such as OCFS2) or high-level CAW commands implemented using the Small Computer System Interface (SCSI), also known as the VMware Virtual Machine File System (VMFS). The former establishes socket connections between nodes to exchange messages, creating communication channels over an existing network. Its stability is affected by network fluctuations, especially in environments without dedicated communication networks, where any network fluctuations or stress can impact cluster stability. While the latter is unaffected by network fluctuations, the SCSI CAW commands are based on the SCSI protocol, which is not supported by low-end or older storage servers, limiting its application scope.

[0004] Therefore, how to balance the above two aspects—being independent of the network and not being subject to specific storage—is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, and medium for accessing a cluster file system, which achieves both independence from the network and freedom from specific storage, thereby improving the stability and scope of use of cluster files.

[0006] To address the aforementioned technical problems, this invention provides a method for accessing a clustered file system, comprising:

[0007] Obtain the metadata blocks of the cluster file system and the corresponding preset nodes of the cluster, where there are multiple preset nodes;

[0008] Add a lock resource to the metadata block and divide the server's disk area according to the lock resource;

[0009] The disk area is divided into multiple subspaces and the corresponding subspaces are allocated according to preset nodes;

[0010] When an IO request is received, the target node is determined by competing for lock resources among preset nodes according to the Paxos algorithm in order to access the lock requested.

[0011] Preferably, the server's disk area is divided according to lock resources, including:

[0012] Get the preset number of nodes corresponding to the cluster and the maximum number of nodes supported by the cluster;

[0013] The server's disk is divided according to the preset number of nodes and the maximum number of nodes to obtain the disk area corresponding to the locked resources.

[0014] Preferably, the allocation of corresponding subspaces according to preset nodes includes:

[0015] In multiple subspaces, the first subspace is used as the state space of the lock resource;

[0016] In the subspaces other than the state space, corresponding subspaces are assigned according to preset nodes, with one preset node corresponding to one subspace.

[0017] Preferably, the subspace stores the voting information of preset nodes.

[0018] Preferably, the target node is determined by competing for lock resources among preset nodes using the Paxos algorithm, including:

[0019] Obtain voting information from each preset node;

[0020] The target node is determined by voting on each preset node based on the voting information and the Paxos algorithm;

[0021] Acquire lock resources based on the target node.

[0022] Preferably, it further includes:

[0023] Set corresponding heartbeat data for each preset node;

[0024] The node status of each preset node is determined based on heartbeat data;

[0025] When the target node is in a working state and the preset time has expired, the target node will be cleaned up and the lock resources will be released so that other nodes can acquire the lock resources.

[0026] Preferably, it further includes:

[0027] When the target node finishes releasing the lock resource, a prompt message is output.

[0028] To address the aforementioned technical problems, the present invention also provides an access device for a clustered file system, comprising:

[0029] The acquisition module is used to acquire the metadata blocks of the cluster file system and the preset nodes corresponding to the cluster, wherein there are multiple preset nodes;

[0030] The partitioning module is used to add lock resources to metadata blocks and partition the server's disk areas according to the lock resources;

[0031] The allocation module is used to divide the disk area into multiple subspaces and allocate the corresponding subspaces according to preset nodes;

[0032] The determination module is used to determine the target node by competing for lock resources among preset nodes according to the Paxos algorithm when an IO request is received, so as to realize the access of the lock request.

[0033] To address the aforementioned technical problems, the present invention also provides an access device for a clustered file system, comprising:

[0034] Memory, used to store computer programs;

[0035] A processor is used to implement the access methods of a clustered file system as described above when executing a computer program.

[0036] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the access method of the cluster file system described above.

[0037] This invention provides a method for accessing a clustered file system, comprising: acquiring metadata blocks of the clustered file system and preset nodes corresponding to the cluster, wherein there are multiple preset nodes; adding lock resources to the metadata blocks and dividing the server's disk area according to the lock resources; dividing the disk area into multiple subspaces and allocating corresponding subspaces according to the preset nodes; when an IO request is received, competing for lock resources among the preset nodes using the Paxos algorithm to determine the target node and realize the lock request access. This method uses the Paxos algorithm to determine the target node and realize the lock request access based on data IO operations of the block file, achieving node-to-node access without relying on the network. IO operations are not limited by specific storage through read and write operations. The added lock area is used for inter-node competition for lock resources. The disk area is divided according to the layout for metadata structures requiring concurrent access protection. Then, based on the Paxos algorithm, IO requests are directly issued through the kernel block layer interface so that the target node can compete for lock resources. This avoids the problem that existing clustered file systems cannot simultaneously achieve network independence and storage independence during concurrent access, improving the stability and usability of clustered files.

[0038] In addition, the present invention also provides an access device and medium for a clustered file system, which has the same beneficial effects as the access method for the clustered file system described above. Attached Figure Description

[0039] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart illustrating a cluster file system access method provided in an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of the disk layout of a cluster file system metafile provided in an embodiment of the present invention;

[0042] Figure 3 This is a structural diagram of an access device for a cluster file system provided in an embodiment of the present invention;

[0043] Figure 4 This is a structural diagram of another cluster file system access device provided in an embodiment of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0045] The core of this invention is to provide a method, apparatus, and medium for accessing a cluster file system, which achieves both independence from the network and freedom from specific storage, thereby improving the stability and scope of use of cluster files.

[0046] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] Existing solutions based on "message passing" strategies rely on message communication between nodes within the cluster. Their main advantage is that they are not dependent on specific storage. However, the stability of the centralized file system in this approach depends on the stability of the existing network; any network fluctuations or stress can affect the cluster's stability. Virtual Virtual File System (VMFS), implemented using the CAW (Carry-Only Wrap) command provided by SCSI, relies on the SCSI CAW lock semantics provided by the storage device. Its advantage is that it only depends on the storage link and can share the storage link with the data channel, making it unaffected by network fluctuations. Generally, storage links have dedicated communication links, resulting in better stability than general-purpose networks. However, this approach also has significant drawbacks, namely its dependence on specific storage devices. The SCSI CAW command was proposed in the SCSI protocol SPC-3; low-end storage or older storage servers do not support the SCSI CAW command, thus limiting the application scope of the cluster file system to specific hardware. It should be noted that the cluster file system access method provided in this invention is generally applicable to cluster file systems, including common cluster file systems such as GFS, VMFS, OpenVMS Files, and ocfs2. This invention is applicable to all of them, is not limited by dedicated communication network environments, and does not depend on specific storage.

[0048] Figure 1 A flowchart illustrating a cluster file system access method provided in an embodiment of the present invention is shown below. Figure 1 As shown, the method includes:

[0049] S11: Obtain the metadata blocks of the cluster file system and the preset nodes corresponding to the cluster, where there are multiple preset nodes;

[0050] Data in any file system is divided into data and metadata. Data refers to the actual data in a regular file, while metadata refers to system data that describes the characteristics of a file, such as access permissions, file owner, and the distribution information of file data blocks (inodes, etc.). In a clustered file system, distribution information includes the file's location on the disk and the disk's location within the cluster. Users need to obtain the metadata of a file before they can locate its position and retrieve its content or related attributes.

[0051] There are two approaches to metadata management: centralized and distributed. Centralized management involves a single node dedicated to metadata management, with all metadata stored on that node's storage. All client requests for files must first request metadata from this metadata manager. Distributed management involves storing metadata on any node in the system and allowing for dynamic migration. The responsibility for metadata management is also distributed across different nodes. Most clustered file systems use centralized metadata management because it is simple to implement, easy to maintain consistency, and can provide satisfactory performance within a certain level of operation frequency. The drawback is the single point of failure problem; if this server fails, the entire system will malfunction. Furthermore, when metadata operations are too frequent, centralized metadata management becomes the performance bottleneck of the entire system.

[0052] Compared to centralized management, distributed metadata management solves the problem of a single point of failure, and its performance does not become a bottleneck with frequent operations. This invention is based on an access method using distributed metadata management.

[0053] A cluster typically consists of multiple nodes. Initially, each node is independent and exists within its own centralized domain. These independent nodes can be connected to form a cluster. A node is essentially a server running in cluster mode.

[0054] S12: Add lock resources to the metadata block and divide the server's disk area according to the lock resources;

[0055] It's important to note that ensuring eventual data consistency requires various technical solutions, such as distributed transactions and distributed locks. To guarantee that a method or property can only be executed by one thread at a time under high concurrency, in traditional monolithic applications deployed on a single machine, concurrency-related functionalities can be used for mutual exclusion control. However, as business needs evolve, the original monolithic, single-machine deployment system evolves into a distributed cluster system. Because distributed systems involve multiple threads and processes distributed across different machines, cross-machine mutual exclusion mechanisms are needed to control access to shared resources.

[0056] Locks for different services are distinguished by different space names. Lock space names differ on different metadata services. The space names for metadata locks and data locks are also different. Theoretically, clients will send requests to all services, and the clients use space names to organize different lock chains.

[0057] The distributed lock module operates relatively independently from other modules in the metadata server, storing all locks that have been requested from the metadata server. When a new lock request arrives, it directly enters the lock request module. The lock module checks whether a lock already exists on the resource and whether there are any lock conflicts to determine whether the requested lock will be granted.

[0058] A lock resource is the object that the lock protects. In addition to files, it can also be an object ID, etc. A lock resource corresponds to an actual entity, such as a file, a piece of data, a database, etc. The object of the lock resource also determines the granularity of the lock. For example, you can lock the entire metadata block or lock an attribute in the metadata block. The former is obviously coarser in granularity than the latter.

[0059] When an application first requests to lock a resource, the lock manager creates a lock resource. The lock resource is associated with the actual resource. A lock resource may be associated with multiple locks, but a lock can only be associated with one lock resource.

[0060] Each lock resource will be allocated a disk area. Due to the redesign of the disk layout of the cluster file system metadata files based on the kernel block layer interface, the metadata structure that requires concurrent access protection will be used for the lock area for inter-node competition for lock resources.

[0061] Regarding how to divide the disk area based on the lock resource, it can be based on the attributes of the lock resource or on the attributes of the cluster node targeted by the lock resource. This invention does not impose specific limitations.

[0062] Distributed locks can be classified in several ways, including mutex locks and shared locks. Mutex locks further include unfair locks and fair locks. A mutex lock allows only one request to hold the resource at a time, regardless of whether the request is for reading or writing. An unfair lock allows only one thread to hold a resource at a time. When another thread needs to acquire the resource, it must listen and wait. Only after the thread holding the resource releases it can other threads compete for it. A fair lock uses a rule-based approach to acquire the lock resource. For example, it checks if the node is the smallest node; if so, it acquires the lock. If not, it listens for lock requests from the previous node, processes them, releases the lock, and then notifies the next listening node that just acquired the resource. A shared lock allows multiple consecutive read requests to acquire the lock simultaneously.

[0063] S13: Divide the disk area into multiple subspaces and allocate the corresponding subspaces according to the preset nodes;

[0064] The disk area is divided into multiple subspaces, which can be the same or different in size. For ease of division, it is preferable to divide the size of the multiple subspaces into different subspaces and allocate the corresponding subspaces according to the number of preset nodes, that is, each preset node is allocated one subspace.

[0065] S14: When an IO request is received, the target node is determined by competing for lock resources among preset nodes according to the Paxos algorithm in order to realize the lock request access.

[0066] In step S13, each preset node is allocated a subspace. When an IO request is received, the Paxos algorithm is used to compete for lock resources among the preset nodes to determine the target node for lock access. During the competition for lock resources among the preset nodes using the Paxos algorithm, each preset node modifies its own subspace and reads the subspaces of other preset nodes.

[0067] The Paxos algorithm is a message-passing-based consensus algorithm with high fault tolerance. It addresses the problem of how to correctly and quickly reach consensus on certain data in a distributed system. In a Paxos system, all nodes are divided into three categories: proposers, acceptors, and learners. Proposers propose solutions, with proposal information including a proposal number and a proposed value. Acceptors participate in the decision-making process, responding to proposals and accepting them upon request. If a proposal is accepted by an acceptor, it is considered approved. Learners do not participate in the decision-making process; they learn from the latest consensus proposals from proposers or acceptors.

[0068] The Paxos algorithm is used to determine the target node among multiple preset nodes, thereby acquiring lock resources to access the lock request. This improves the stability of hyperconverged and virtualized systems, reduces system costs, and effectively enhances product competitiveness.

[0069] This invention provides a method for accessing a clustered file system, comprising: acquiring metadata blocks of the clustered file system and preset nodes corresponding to the cluster, wherein there are multiple preset nodes; adding lock resources to the metadata blocks and dividing the server's disk area according to the lock resources; dividing the disk area into multiple subspaces and allocating corresponding subspaces according to the preset nodes; when an IO request is received, competing for lock resources among the preset nodes using the Paxos algorithm to determine the target node and realize the lock request access. This method uses the Paxos algorithm to determine the target node and realize the lock request access based on data IO operations of the block file, achieving node-to-node access without relying on the network. IO operations are not limited by specific storage through read and write operations. The added lock area is used for inter-node competition for lock resources. The disk area is divided according to the layout for metadata structures requiring concurrent access protection. Then, based on the Paxos algorithm, IO requests are directly issued through the kernel block layer interface so that the target node can compete for lock resources. This avoids the problem that existing clustered file systems cannot simultaneously achieve network independence and storage independence during concurrent access, improving the stability and usability of clustered files.

[0070] Based on the above embodiments, step S12, which involves dividing the server's disk area according to lock resources, includes:

[0071] Get the preset number of nodes corresponding to the cluster and the maximum number of nodes supported by the cluster;

[0072] The server's disk is divided according to the preset number of nodes and the maximum number of nodes to obtain the disk area corresponding to the locked resources.

[0073] Figure 2 This is a schematic diagram of the disk layout of a cluster file system metafile provided in an embodiment of the present invention, as shown below. Figure 2 As shown, lock resources are added at the metadata block. Taking inode metadata as an example, the lock resources are adjacent to the inode metadata block and occupy one or more filesystem blocks.

[0074] The size of the server's disk area is divided according to the cluster attributes corresponding to the lock resource. The specific number of preset nodes in the cluster is related to the maximum number of nodes supported by the cluster. Metadata blocks can be inodes or dentry, etc., and this invention does not impose specific limitations.

[0075] This invention provides a method for obtaining the preset number of nodes corresponding to a cluster and the maximum number of nodes supported by the cluster; and for partitioning the server's disk according to the preset number of nodes and the maximum number of nodes to obtain the disk region corresponding to the lock resource. Partitioning the lock resource into disk regions facilitates competition for lock resources among nodes.

[0076] Based on the above embodiments, step S13, which allocates the corresponding subspace according to the preset node, includes:

[0077] In multiple subspaces, the first subspace is used as the state space of the lock resource;

[0078] In the subspaces other than the state space, corresponding subspaces are assigned according to preset nodes, with one preset node corresponding to one subspace.

[0079] like Figure 2 As shown, the disk area is divided into multiple subspaces, each with a size of 512K or 4K, without specific limitations. The size can be determined based on the actual situation. It should be noted that when dividing into multiple subspaces, the first subspace (modification node) is used as a state description of the lock resource and is not assigned a preset node. The remaining subspaces are allocated according to a preset node.

[0080] There are four states associated with lock resources: unlocked, biased lock, lightweight lock, and heavyweight lock. As lock contention intensifies, a lock can escalate from a biased lock to a lightweight lock, and then to a heavyweight lock. Each object is initially unlocked; as thread contention for the lock intensifies, the lock level increases, and locks can only be upgraded, not downgraded.

[0081] The lock escalation process: When an object is first created, it is in an unlocked state; when one thread uses the object, it is upgraded to a biased lock and marked as a biased lock in the object header. When the same thread accesses the object again, it uses the biased lock directly; when two threads compete for the object, the lock is upgraded from a biased lock to a lightweight lock; when more threads compete for the object, the lightweight lock is upgraded to a heavyweight lock.

[0082] In this embodiment, the first subspace is used as the state space of the lock resource in multiple subspaces. In the subspaces other than the state space, corresponding subspaces are allocated according to preset nodes, with one preset node corresponding to one subspace, which facilitates the metadata structure for concurrent access protection.

[0083] Based on the above embodiments, the subspace stores the voting information of preset nodes.

[0084] Based on the subsequent adoption of the Paxos algorithm, which mainly determines the target node of each preset node through voting information, the subspace stores voting information and can also store other data information. This invention does not make specific limitations, but at least includes voting information.

[0085] The subspace provided in this embodiment stores the voting information of preset nodes, which facilitates the subsequent determination of the target node to realize the access of the lock request.

[0086] Based on the above embodiments, step S14, which involves determining the target node by competing for lock resources among preset nodes using the Paxos algorithm, includes:

[0087] Obtain voting information from each preset node;

[0088] The target node is determined by voting on each preset node based on the voting information and the Paxos algorithm;

[0089] Acquire lock resources based on the target node.

[0090] Understandably, a proposer proposes a value for voting. In most scenarios, the node in the cluster that receives the client's request is the proposer. This role represents access and coordination; upon receiving the client's request, it initiates a two-phase commit to negotiate consensus. Acceptors vote on each proposed value and store the accepted value, for example, nodes A, B, and C. Generally, all nodes in the cluster act as acceptors, participating in consensus negotiation and accepting and storing data. Acceptors represent voting negotiation and data storage, voting on proposed values ​​and accepting the agreed-upon value, storing and saving it. Learners are informed of the voting results, accept the agreed-upon value, store and save it, and do not participate in the voting process. Generally, learners are data backup nodes, such as the Slave in a "Master-Slave" model, passively accepting data and providing disaster recovery backup. Learners represent data storage, do not participate in consensus negotiation, and only accept and store the agreed-upon value.

[0091] For example, two clients act as initiators, and three nodes act as receivers. Client 1 wants to set the value of data with key X in node 3, and client 2 wants to set the value of data with key X in node 5. The information sent by the initiator to the receiver is called a proposal, with the structure [n, v], where n is the proposal number (equivalent to a transaction ID, with later proposals having larger numbers), and v is the proposal value (the value written to the database). During the preparation phase, the two initiators each send a preparation request containing only the proposal number to the three receivers. Each receiver node responds to the first preparation request it receives. Since no proposals have been approved before, A, B, and C will all return a "No proposals yet" response. However, there is a difference: A and B will tell the initiator that they will no longer respond to preparation requests with proposal numbers less than or equal to 1, and C will tell the initiator that it will no longer respond to preparation requests with proposal numbers less than or equal to 5. In other words, each node will subsequently accept requests with proposal numbers greater than the current proposal number.

[0092] Next, each receiver receives its second response to the preparation request. Requests received by A and B, numbered 5 (greater than or equal to 1), and since neither node has passed any proposals at this point, return a "No proposals yet" response and no longer respond to preparation requests with proposal numbers less than or equal to 5. Requests received by C, numbered 1 (less than 5), are discarded without a response.

[0093] During the acceptance phase, after receiving a majority of ready responses from the nodes, both proposer nodes will send acceptance requests. For client 1, the value in the acceptance request is set according to the value of the proposal with the largest proposal number in the response. (Client 1 only receives ready responses from A and B). Since the responses are all "No proposal yet", client 1 will use its own proposal value: 3 as the proposal value and then send the acceptance request [n,v]:[1,3]. For client 2, the value in the acceptance request is set according to the value of the proposal with the largest proposal number in the response. (Client 2 receives ready responses from A, B, and C). Since the responses are all "No proposal yet", client 1 will use its own proposal value: 7 as the proposal value and then send the acceptance request [n,v]:[5,7].

[0094] The three acceptor nodes receive acceptance requests from two proposers and will process them as follows: For nodes A, B, and C, they will not accept request [1,3] because the proposal number is <5 (the minimum proposal number). They will accept request [5,7] because the proposal number is greater than or equal to 5. After the proposal is approved, the proposal value: 7 will be used as the value of X.

[0095] Based on the proposal number, the acceptor makes three promises: specifically, if the proposal number of the prepared request is less than or equal to the proposal number of a prepared request already responded to by the acceptor, then the acceptor promises not to respond to this prepared request; if the proposal number of the accepted request is less than the proposal number of a prepared request already responded to by the acceptor, then the acceptor promises not to approve this proposal; if the acceptor has previously approved a proposal, then the acceptor promises to include information about the highest approved proposal number in the response to the prepared request. The above describes the Basic Paxos algorithm. In this embodiment of the invention, the Paxos algorithm can be either Basic Paxos or Multi-Paxos, without specific limitation, and can be set according to specific circumstances.

[0096] In the Multi-Paxos algorithm, the leader node acts as the sole proposer, preventing multiple proposers from submitting proposals simultaneously and eliminating proposal conflicts. The master node is elected through a voting process using the Basic Paxos algorithm, and its lease is continuously extended during operation. For example, in real-world scenarios, the same node may serve as the master node for several days. If the master node fails, other nodes will vote to elect a new master node, ensuring the master node is always present and unique. All read and write requests are handled by the master node. When the master node receives a write request from a client, it acts as the proposer, executing a Basic Paxos instance to send the data to all nodes. Only after a majority of servers accept the write request does it respond with a success message to the client. When the master node receives a read request, its processing is simpler; it only needs to query local data and return the result to the client.

[0097] This invention provides a method for obtaining voting information from each preset node, performing voting processing on each preset node based on the voting information and the Paxos algorithm to determine a target node, and acquiring lock resources based on the target node. Resource contention and acquisition are achieved through the logic of the Paxos algorithm.

[0098] Based on the above embodiments, the method further includes:

[0099] Set corresponding heartbeat data for each preset node;

[0100] The node status of each preset node is determined based on heartbeat data;

[0101] When the target node is in a working state and the preset time has expired, the target node will be cleaned up and the lock resources will be released so that other nodes can acquire the lock resources.

[0102] Understandably, with persistent connections, there might be no data exchange for an extended period. Theoretically, the connection should remain open indefinitely, but in reality, it's difficult to detect if a node in the middle fails. More critically, some nodes (firewalls) will automatically disconnect connections that haven't exchanged data for a certain period. In such cases, heartbeats can be used to maintain persistent connections and ensure they remain alive.

[0103] The heartbeat mechanism involves sending a fixed message to the server every few minutes. Upon receiving this message, the server replies with a fixed message. If the server doesn't receive a message from the client within a few minutes, it considers the connection broken. The sender can be either the client or the server, depending on which implementation is more convenient and reasonable.

[0104] Heartbeat sending typically involves two techniques: application-layer heartbeats and the SO_KEEPALIVE socket option. The former involves the application sending heartbeats to check connection health. The server sends a short data packet to the client at regular intervals and starts a thread that continuously monitors the client's response. If no response is received within a certain time, the client is considered disconnected; similarly, if the client doesn't receive a heartbeat from the server within a certain time, the connection is considered unavailable. The latter, however, is an inherent heartbeat mechanism within TCP itself, implemented as a TCP option. Whether on the server or client side, once keep-alive is enabled, it automatically sends a heartbeat to the other side at specified intervals. The other side, upon receiving the heartbeat, automatically replies to indicate that it remains online.

[0105] Each preset node sets heartbeat data to know its node status when it is read by other preset nodes. When the target node's node status is working and exceeds the preset time, it means that the target node may be offline. The target node is then cleaned up and the lock resources are released so that other nodes can acquire the lock resources.

[0106] Specifically, in offline scenarios, the target node may be recovering from a node crash or I / O failure, and its status can be determined through heartbeat data. When the target node is offline, to improve the access efficiency of lock requests, another target node can be selected from the remaining nodes using the Paxos algorithm to compete for lock resources.

[0107] This embodiment provides setting corresponding heartbeat data for each preset node; determining the node status of each preset node based on the heartbeat data; when the target node's node status is working and exceeds a preset time, the target node is cleaned up and its lock resources are released so that other nodes can acquire lock resources, improving the access efficiency of lock requests, and is used for the allocation and release of lock space so that other nodes can compete to acquire lock resources.

[0108] Based on the above embodiments, the method further includes:

[0109] When the target node finishes releasing the lock resource, a prompt message is output.

[0110] Understandably, to allow other nodes to acquire the lock resource as quickly as possible, a prompt message is output when the target node releases the lock resource. This prompt message can take the form of voice, a pop-up window on the user interface, the result of program execution, or a combination thereof.

[0111] This invention provides a method to output a prompt message when the target node finishes releasing the lock resource. This promptly reminds the user to request locks on other nodes, improving access efficiency.

[0112] The foregoing has described in detail various embodiments corresponding to the cluster file system access methods. Based on this, the present invention also discloses a cluster file system access device corresponding to the above methods. Figure 3 This is a structural diagram of an access device for a clustered file system provided in an embodiment of the present invention. Figure 3 As shown, the access devices for the cluster file system include:

[0113] The acquisition module 11 is used to acquire the metadata blocks of the cluster file system and the preset nodes corresponding to the cluster, wherein there are multiple preset nodes;

[0114] The partitioning module 12 is used to add lock resources to the metadata blocks and partition the server's disk areas according to the lock resources;

[0115] Allocation module 13 is used to divide the disk area into multiple subspaces and allocate the corresponding subspaces according to preset nodes;

[0116] The determination module 14 is used to determine the target node by competing for lock resources among preset nodes according to the Paxos algorithm when an IO request is received, so as to realize the access of the lock request.

[0117] Since the embodiments of the device part correspond to the embodiments described above, please refer to the embodiments described in the method part for the embodiments of the device part, and will not be repeated here.

[0118] This invention provides an access device for a clustered file system, comprising: acquiring metadata blocks of the clustered file system and preset nodes corresponding to the cluster, wherein there are multiple preset nodes; adding lock resources to the metadata blocks and dividing the server's disk area according to the lock resources; dividing the disk area into multiple subspaces and allocating corresponding subspaces according to the preset nodes; when receiving an IO request, determining the target node by competing for lock resources among the preset nodes according to the Paxos algorithm to realize the lock request access. This device uses the Paxos algorithm to determine the target node for lock request access based on data IO operations of the block file, achieving node-to-node access without relying on the network. IO operations are not limited by specific storage through read and write operations. Read and write operations are used to compete for lock resources among nodes based on the added lock area. Disk areas are divided according to the layout for metadata structures requiring concurrent access protection. Then, based on the Paxos algorithm, IO requests are directly issued through the kernel block layer interface so that target nodes can compete for lock resources. This avoids the problem that existing clustered file systems cannot simultaneously achieve network independence and storage independence during concurrent access, improving the stability and usability of clustered files.

[0119] Figure 4 A structural diagram of another cluster file system access device provided in an embodiment of the present invention is shown below. Figure 4 As shown, the device includes:

[0120] Memory 21 is used to store computer programs;

[0121] Processor 22 is used to implement the steps of accessing the clustered file system when executing a computer program.

[0122] The access device for the cluster file system provided in this embodiment may include, but is not limited to, smartphones, tablets, laptops, or desktop computers.

[0123] The processor 22 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 22 may be implemented using at least one of the following hardware forms: Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 22 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 22 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 22 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.

[0124] The memory 21 may include one or more computer-readable storage media, which may be non-transitory. The memory 21 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 21 is used to store at least the following computer program 211, which, after being loaded and executed by the processor 22, is capable of implementing the relevant steps of the cluster file system access method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 21 may also include an operating system 212 and data 213, etc., and the storage method may be temporary storage or permanent storage. The operating system 212 may include Windows, Unix, Linux, etc. The data 213 may include, but is not limited to, the data involved in the cluster file system access method, etc.

[0125] In some embodiments, the access device for the cluster file system may further include a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27.

[0126] Those skilled in the field can understand, Figure 4 The structure shown does not constitute a limitation on the access device for the cluster file system and may include more or fewer components than illustrated.

[0127] The processor 22 implements the cluster file system access method provided in any of the above embodiments by calling instructions stored in the memory 21.

[0128] This invention provides an access device for a clustered file system, comprising: acquiring metadata blocks of the clustered file system and preset nodes corresponding to the cluster, wherein there are multiple preset nodes; adding lock resources to the metadata blocks and dividing the server's disk area according to the lock resources; dividing the disk area into multiple subspaces and allocating corresponding subspaces according to the preset nodes; when receiving an IO request, determining the target node by competing for lock resources among the preset nodes according to the Paxos algorithm to realize the lock request access. This device uses the Paxos algorithm to determine the target node for lock request access based on data IO operations of the block file, achieving node-to-node access without relying on the network. IO operations are not limited by specific storage through read and write operations. Read and write operations are used to compete for lock resources among nodes based on the added lock area. Disk areas are divided according to the layout for metadata structures requiring concurrent access protection. Then, based on the Paxos algorithm, IO requests are directly issued through the kernel block layer interface so that target nodes can compete for lock resources. This avoids the problem that existing clustered file systems cannot simultaneously achieve network independence and storage independence during concurrent access, improving the stability and usability of clustered files.

[0129] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by processor 22, implements the steps of the access method of the cluster file system described above.

[0130] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0131] For an introduction to the computer-readable storage medium provided by the present invention, please refer to the above method embodiments. The present invention will not be described in detail here, but it has the same beneficial effects as the above-described cluster file system access method.

[0132] The foregoing has provided a detailed description of a cluster file system access method, access device, and medium provided by the present invention. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0133] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. An access method of a clustered file system, characterized by, The application relates to a method for realizing a lock resource of a cluster file system. The method comprises the following steps: acquiring a preset node corresponding to a metadata block of the cluster file system, wherein the preset node is multiple; increasing a lock resource in the metadata block and dividing a disk area of a server according to the lock resource; specifically, the preset node number corresponding to the cluster and the maximum node number supported by the cluster are acquired; the disk of the server is divided according to the preset node number and the maximum node number to obtain the disk area corresponding to the lock resource; the disk area is divided into multiple subspaces, and the corresponding subspaces are allocated according to the preset nodes; specifically, in the multiple subspaces, the first subspace is used as a state space of the lock resource; in the subspaces except the state space, the corresponding subspaces are allocated according to the preset nodes, wherein one preset node corresponds to one subspace; the first subspace is used for modifying nodes, and the state of the lock resource comprises a lock-free state, a bias lock, a light-weight lock and a heavy-weight lock; a lock upgrade process: an object is in a lock-free state when being created; when one thread uses the object, the lock is upgraded to a bias lock, and the bias lock is marked in the object header; when the thread accesses the object next time, the object is directly used; when two threads compete for the object, the lock is upgraded from the bias lock to a light-weight lock; when more than two threads compete for the object, the light-weight lock is upgraded to a heavy-weight lock; 2. The access method of a clustered file system according to claim 1, wherein, when an IO request is received, the preset nodes are competed for the lock resource according to a Paxos algorithm to determine a target node to realize access of the lock request.

3. The access method of a clustered file system according to claim 2, wherein, The subspaces store voting information of the preset nodes. The preset nodes are competed for the lock resource according to the Paxos algorithm to determine the target node, which comprises the following steps: acquiring the voting information of each preset node; determining the target node by voting each preset node according to the voting information and the Paxos algorithm; 4. The access method of a clustered file system according to any one of claims 1 to 3, wherein, acquiring the lock resource according to the target node. The application further comprises the following steps: setting corresponding heartbeat data for each preset node; determining the node state of each preset node according to the heartbeat data; 5. The access method of a clustered file system according to claim 4, wherein, when the node state of the target node is a working state and exceeds a preset time, the target node is cleaned up and the lock resource is released so that the remaining nodes acquire the lock resource. The application further comprises the following step:

6. An access device of a clustered file system, characterized by, when the target node releases the lock resource, outputting prompt information. The application relates to a method for realizing a lock resource of a cluster file system. The method comprises the following steps: an acquisition module is used for acquiring a preset node corresponding to a metadata block of the cluster file system, wherein the preset node is multiple; a division module is used for increasing a lock resource in the metadata block and dividing a disk area of a server according to the lock resource; specifically, the preset node number corresponding to the cluster and the maximum node number supported by the cluster are acquired; the disk of the server is divided according to the preset node number and the maximum node number to obtain the disk area corresponding to the lock resource; The allocation module is used for cutting the disk area into a plurality of subspaces and allocating corresponding subspaces according to the preset nodes; specifically comprising: in a plurality of the subspaces, the first subspace is used as the state space of the lock resource; in the subspaces except the state space, corresponding subspaces are allocated according to the preset nodes, wherein one of the preset nodes corresponds to one of the subspaces; the first subspace is used for modifying nodes, and the state of the lock resource includes a lock-free state, a biased lock, a lightweight lock and a heavy lock; a lock upgrade process: an object is in a lock-free state when it is just created; when one thread uses the object, the lock is upgraded to a biased lock, which is marked in the object header as a biased lock, and when the thread accesses the object next time, the object is directly used; when two threads compete for the object, the lock is upgraded from the biased lock to the lightweight lock; when more than two threads compete for the object, the lightweight lock is upgraded to the heavy lock; The determination module is used for determining a target node for the lock resource competition of the preset nodes according to the Paxos algorithm when an IO request is received, so as to realize the access of the lock request.

7. An access device of a clustered file system, characterized by, Comprise: A memory for storing a computer program; A processor for executing the computer program to realize the steps of the access method of the cluster file system according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the steps of the access method of the cluster file system according to any one of claims 1 to 5.

Images