File lock implementation method, device and equipment in distributed file system

Abstract

The invention provides a method, a device and equipment for realizing a file lock in a distributed file system, which are used for solving the technical problem that the consistency of the file lock in the distributed file system is difficult to guarantee. According to the invention, the remote procedure call state monitoring service and the network state monitoring notification service function of the NFS server are combined into the NFS service process, so that the file lock processing efficiency and reliability are improved. When the client file lock request is processed, the client information is recorded in the distributed storage system by calling the underlying file system interface, so that the problem that a new node of an NFS server cannot acquire the client information of the file lock in an abnormal switching scene of the NFS server is solved. According to the technical scheme provided by the invention, the consistency of the file lock in the distributed system can be effectively ensured, the lock state recovery process is simplified, and the implementation efficiency and reliability of the file lock are improved.

Description

Technical Field

[0001] The present invention relates to the field of distributed storage technology, and in particular to a method, device and equipment for implementing file locks in a distributed file system. Background Art

[0002] Distributed storage systems are widely favored by the market due to their scalability, high reliability, and high performance. Among them, Ceph is widely used as a unified storage that supports block storage, object storage, and file storage. Network File System (NFS) is a network file system with distributed characteristics and supports standard POSIX file semantics. Therefore, as an access protocol, NFS can deploy a variety of upper-layer applications based on the NFS file system. Database business is a relatively common upper-layer access application. Database applications not only have basic read and write requirements, but also have a large number of file lock requests.

[0003] For the NFSv3 file lock service, after the client sends a file lock request, the NFSv3 Network Lock Manager (NLM) first sends the request to the NFS server. The NFS server processes the request and calls the file lock interface of the underlying file system to implement the file lock semantics.

[0004] The current NFS file lock processing mechanism has pain points in distributed storage systems, such as difficulty in ensuring lock consistency and complicated lock status recovery. Summary of the invention

[0005] In view of this, the present invention provides a method, apparatus and device for implementing file lock in a distributed file system, which are used to solve the technical problem that the consistency of file lock in a distributed storage system is difficult to ensure.

[0006] Based on one aspect of the present invention, the present invention provides a method for implementing file lock in a distributed file system, the method comprising:

[0007] After the distributed file system NFS server receives the lock request sent by the NFS client, the internally integrated remote procedure call status monitoring service calls the underlying file system interface to record the client monitoring information in the lock request into the distributed file system;

[0008] The NFS server calls the interface provided by the underlying file system to process the lock request of the NFS client and perform the file lock operation;

[0009] The NFS server feeds back the lock request processing result to the NFS client.

[0010] Furthermore, the method further comprises:

[0011] After receiving the unlock request from the NFS client, the NFS server calls the interface provided by the underlying file system to unlock the file stored in the distributed file system;

[0012] After the NFS server receives a successful response to the file unlocking, the internally integrated remote procedure call status monitoring service calls the interface provided by the underlying file system to delete the client monitoring information stored in the distributed file system;

[0013] The NFS server feeds back the result of the file unlock request to the NFS client.

[0014] Furthermore, the method further comprises:

[0015] In the scenario where NFS server switching occurs, the destination NFS server takes over the services of the source destination NFS server;

[0016] When the NFS server switch is completed, the destination NFS server calls the interface provided by the underlying file system to read the client monitoring information created by the source NFS server from the distributed file system;

[0017] The destination NFS server sends a NOTIFY message to the NFS client based on the read client monitoring information, so that the NFS client replays the acquired file lock.

[0018] Furthermore, the method further comprises:

[0019] The client monitoring information is stored in the metadata service MDS in the distributed file system;

[0020] In the MDS abnormal switching scenario, the switching target MDS reads the client monitoring information stored in the persistent storage medium in the distributed storage cluster, rebuilds the client monitoring information and provides file lock services for the upper layer.

[0021] Furthermore, the NFS server integrates not only the remote procedure call monitoring service (rpc.statd) process but also the network status monitoring notification service (sm-notify) process.

[0022] Based on another aspect of the present invention, the present invention further provides a file lock implementation device in a distributed file system, the device is applied to a distributed file system NFS server, the device includes:

[0023] The file lock monitoring information processing module is used to, after receiving the lock request sent by the NFS client, call the underlying file system interface by the internally integrated remote procedure call status monitoring service to record the client monitoring information in the lock request into the distributed file system;

[0024] The file lock execution module is used to call the interface provided by the underlying file system, process the lock request of the NFS client, and execute the file lock operation;

[0025] The lock request response module is used to feedback the lock request processing result to the NFS client.

[0026] Furthermore, the file lock execution module is further used to call the interface provided by the underlying file system to unlock the file stored in the distributed file system after receiving the unlock request from the NFS client and determining that the file is the last file lock of the NFS client;

[0027] The file lock monitoring information processing module is further used to delete the client monitoring information stored in the distributed file system by calling the interface provided by the underlying file system through the internally integrated remote procedure call status monitoring service after receiving a successful file unlock response;

[0028] The lock request response module is also used to feed back the file unlock request processing result to the NFS client.

[0029] Furthermore, in the scenario where NFS server switching occurs, the switched destination NFS server takes over the services of the source destination NFS server;

[0030] The file lock monitoring information processing module is also used to call the interface provided by the underlying file system to read the client monitoring information created by the source NFS server from the distributed file system after the NFS server switching is completed;

[0031] The file lock monitoring information processing module is further used to send a NOTIFY message to the NFS client based on the read client monitoring information, so that the NFS client replays the acquired file lock.

[0032] Furthermore, the NFS server integrates not only the remote procedure call monitoring service (rpc.statd) process but also the network status monitoring notification service (sm-notify) process.

[0033] The device provided by the present invention can be implemented in software, hardware or a combination of software and hardware. When implemented in a software module, the program code of the software module is loaded into the storage medium of the device, and the processor reads the program code in the storage medium and executes it. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments of the present invention or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings of the embodiments of the present invention.

[0035] Figure 1 This is a schematic diagram of the distributed storage system module architecture;

[0036] Figure 2 This is a schematic diagram of the timing flow of NFS client file lock request and NFS server corresponding processing;

[0037] Figure 3 A flowchart of the logic and usage scenarios for granting NLM file locks;

[0038] Figure 4 This is a schematic diagram of the lock processing flow in the NFS client abnormal scenario;

[0039] Figure 5 This is a schematic diagram of the lock processing process when the NFS server crashes or loses power;

[0040] Figure 6 A schematic flow chart of steps of a method for implementing a distributed file system file lock is provided for one embodiment of the present invention;

[0041] Figure 7 A schematic diagram of a file lock request and unlock request processing flow provided by an embodiment of the present invention;

[0042] Figure 8 A schematic diagram of a process flow for triggering file lock replay when an NFS server switch occurs after an NFS server crashes in one embodiment of the present invention;

[0043] Fig. 9 This is a schematic diagram of the MDS abnormal switching scenario processing flow in one embodiment of the present invention;

[0044] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention for implementing a file lock implementation method in a distributed file system provided in the present invention. DETAILED DESCRIPTION

[0045] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, rather than limiting the embodiments of the present invention. The singular forms of "a", "said" and "the" used in the embodiments of the present invention are also intended to include plural forms, unless the context clearly indicates other meanings. Although the embodiments of the present invention may be described in a first, second, or third manner, this description is only used to distinguish similar information, entities, or steps, rather than to describe a specific order or sequence. For example, without departing from the scope of the embodiments of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. For another example, in some scenarios, the first information may refer to one information or to multiple information of the same type. In addition, the word "if" used may be interpreted as "at..." or "when..." or "in response to determination". The "and / or" in the present invention is only a kind of association relationship describing the associated objects, indicating that there may be three relationships, for example, A and / or B, which may represent: A exists alone, A and B exist at the same time, and B exists alone, wherein A and B may be singular or plural. In addition, in the description of the present invention, unless otherwise specified, "multiple" refers to two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0046] Figure 1 The figure is a schematic diagram of the distributed storage system module architecture. In the distributed storage system module architecture based on Ceph, the NFS client accesses the file data in the distributed storage system by calling the lib library / posix library provided by the underlying file system through the NFS server. The distributed storage system processes data and metadata separately. The data is stored in the object storage device (OSD), and the metadata is stored in the metadata service (MDS). The distributed storage system supports object, block and file storage types. The file data will eventually be divided into data objects and stored in the OSD, and the file metadata will be stored in the MDS.

[0047] For the file lock service of Network File System Protocol Version 3 (NFSv3), after the client sends a file lock request, the request is first sent to the NFS server through the NFSv3 Network Lock Manager (NLM). The NFS server processes the request and calls the file lock interface provided by the underlying file system to implement the file lock semantics.

[0048] NFSv3 file locks are implemented by multiple protocols, including NFSv3, PORTMAP, NLM, and the network status monitoring protocol NSM. Multiple protocols work together to ensure lock consistency. However, due to the large number of uncontrollable factors between the NFS client and the NFS server, it is particularly difficult to ensure lock consistency for distributed systems, as the server involves synchronization between multiple nodes. NFSv3 file locks are permanent locks, that is, after the client initiates a lock request and the server successfully grants the lock, the lock will exist permanently until the client actively unlocks it. This puts forward a series of requirements for distributed storage systems. For example, when the client holds the lock, the server switches its business, and the new node must ensure that it has the same file lock status as the old node. Generally speaking, in scenarios such as server business switching, the server actively sends a notification request to the client through the NSM protocol, and the client sends a request to rebuild the file lock to achieve file lock consistency. At the same time, in order to ensure that the client successfully rebuilds the file lock, the server must enter a grace period, during which only requests to rebuild the file lock are processed and non-rebuilt file lock requests are rejected. This will cause upper-layer applications that rely on file locks to be unavailable during the grace period.

[0049] The NSM protocol is a server / client peer-to-peer protocol. Both the server and the client use this protocol to record the information of the other party and persist it in a local file. The file name is the IP of the other party. The file content records the NLM protocol program number, version number, process number, timestamp-related custom information, as well as the IP of the other party and the host name of the local party. Regardless of whether the server or the client crashes, the file is read by the NSM protocol after crash recovery to inform the other party that the local party has crashed in order to restore the file lock status. For the server, after crash recovery, the client is informed, and the client replays all file locks it holds and restores the file lock status. For the client, after crash recovery, the server is informed, and the server learns that the client has crashed and releases all file locks held by the client.

[0050] Currently, Ceph-based distributed storage systems access the NFS protocol and support NFSv3 file locks. Usually, the user-mode NFS protocol is implemented by the NFS file server (such as NFS-Ganesha). NFS-Ganesha implements the NLM protocol internally and can process file lock requests from NFS clients.

[0051] Figure 2 The figure is a flowchart of the NFS client file lock request and the corresponding processing by the NFS server.

[0052] based on Figure 2 For example, in the current lock implementation process, the NFS client processes a complete file lock request as follows:

[0053] Step 01. The NFS client upper layer application calls fcntl to initiate a lock request;

[0054] Step 02. The lockd module in the NFS client kernel communicates with the local rpc.statd (Remote Procedure Callstatus daemon) and uses the NSM protocol to monitor MONITOR requests to create a monitoring information file named NFS server IP. This file is a persistent basis for the client to record which NFS server it holds the file lock on.

[0055] Step 03. The NFS client initiates a locking RPC request to the NFS server.

[0056] Step 04. The NFS server processes the RPC request for locking the NFS client. It first parses the client information, encapsulates the client information into an NSM protocol monitoring MONITOR request data packet and sends it to the local rpc.statd. rpc.statd creates a client monitoring information file (including the client IP address, etc.). This file is the basis for notifying which clients to replay the file lock after the NFS server crashes and recovers.

[0057] Step 05. The NFS server calls the POSIX interface provided by the underlying file system to process the lock request from the NFS client.

[0058] Step 06. The NFS server returns the NFS client lock request processing result to the NFS client.

[0059] In the above process, step 02 and step 04 are specific embodiments of the NSM protocol.

[0060] based on Figure 2 For example, in the current lock implementation, the NFS client processes a complete file lock unlock request as follows:

[0061] Step 07. The NFS client upper layer application calls fcntl to initiate an unlock request.

[0062] Step 08. The NFS client initiates an unlock RPC request to the NFS server.

[0063] Step 09. The NFS server processes the RPC request for unlocking from the NFS client, calls the posix interface provided by the underlying file system, and processes the unlocking request from the NFS client.

[0064] Step 10. After the NFS server calls the underlying file system posix interface to complete the unlocking and returns the unlocking success, it parses the client information. The NFS server determines whether it is the unlocking request of the last lock of the NFS client based on the lock count. If it is determined to be the last lock, the client information is encapsulated into the NSM protocol to monitor the UNMONITOR request data packet and send it to the local rpc.statd. rpc.statd deletes the client monitoring information file named as the client IP, indicating that the client no longer holds the file lock.

[0065] Step 11. The NFS server feeds back the unlock request processing result to the NFS client.

[0066] Step 12. The lockd module in the NFS client kernel communicates with the local rpc.statd and deletes the monitoring information file named NFS server IP through the NSM protocol UNMONITOR request (when there is no lock), indicating that the client no longer holds the file lock on this NFS server.

[0067] NLM file locks have blocking semantics, that is, after the client initiates a lock request, if it is a blocking file lock, when the lock cannot be granted, it will wait synchronously until the lock can be granted. For the server, when processing a blocking file lock request, the server needs to return the blocking lock grant result to the client first. If it cannot be granted immediately, it will be granted asynchronously. Once the grant is successful, it is necessary to actively notify the client that the file lock has been granted, completing the asynchronous grant process.

[0068] Figure 3 A flowchart of the logic and usage scenarios of NLM file lock granting. The SQL application on NFS client A performs an insert operation on the database table. The SQL application sends a LOCK request to the database file A. After the LOCK request is successfully processed by the NFS server and the file lock is granted, the subsequent insert operation, that is, the file write operation, is performed. Before NFS client A completes the data table insert and releases the file lock of database file A, NFS client B cannot insert at the same time. Because the prerequisite for writing the database file - obtaining the file lock of database file A cannot be completed, there is a file lock conflict with NFS client A. NFS client B can perform SQL insert operations only until NFS client A releases the file lock of database file A. The SQL application ensures the consistency of the database table through the file lock mechanism.

[0069] Figure 4The figure is a diagram of the lock processing flow in the NFS client exception scenario. Since the NLM file lock is a permanent lock, once the NFS client crashes after locking, it cannot actively release the lock. For example, if NFS client A crashes after successfully locking, the lock granted to the client will remain permanently, and other clients will not be able to obtain the lock. For example, if NFS client B locks the same file after NFS client A crashes, the NFS server will feedback a conflict (CONFLICT) response. Therefore, once the NFS client crashes, after the NFS client recovers, the NSM protocol needs to send a notify request to the NFS server to inform the server that all file locks of the client need to be released. After the release, other clients can continue to lock.

[0070] Figure 5 The figure is a schematic diagram of the lock processing process when the NFS server crashes or loses power. When multiple NFS servers provide services, after NFS client A successfully locks file A through NFS server A, if NFS server A fails and causes the system to crash, due to the distributed characteristics, in order to achieve high availability, a switch usually occurs, and the NFS server B on the new normal node takes over the business of NFS server A on the old abnormal node. Ideally, after the new node takes over the business of the old node, it needs to send a notify message to the NFS client that has been granted a lock by the original old node to notify the NFS client to rebuild the file lock. However, the actual situation is that the client monitoring information is recorded by the rpc.statd process of the old node, and the recorded client monitoring information is only persisted locally. After the business is switched to the new node, the new node cannot obtain the client monitoring information associated with the old node, that is, NFS server B cannot obtain the persistent client monitoring information on NFS service A, and cannot send the NOTIFY message of the NSM protocol to the corresponding client A to inform the client to rebuild the file lock. At the same time, since the client has obtained the file lock through the old node, the underlying file system still retains the file lock. The new node cannot rebuild the existing file lock, and the client's unlock request cannot be responded to, which will cause the file lock to remain permanently. Other clients will not be able to obtain the lock, affecting the business.

[0071] From the above introduction to file locks, we can see that the implementation process of NFS file locks in the current distributed storage system has the following problems:

[0072] (1) The file lock service involves the interaction of multiple protocols between multiple processes, and the exception handling is extremely complex. The file lock service requires the interaction between at least eight processes, including the NFS client nfs client process, rpc.statd process, sm-notify process, and the NFS server rpcbind process, NFS-Ganesha process, rpc.statd process, sm-notify process, and MDS process. The protocols involved include NFSv3, PORTMAP, NLM, and NSM. Once any of the processes is abnormal, the file lock service becomes unreliable and the consistency of the lock cannot be guaranteed.

[0073] (2) The client information is recorded by rpc.statd and persisted as a local file, which can easily cause service unavailability when a service switch occurs on the server. The NFS server grants the NFS client a file lock and switches abnormally. The new node that takes over the service relies on the client to rebuild the file lock, but cannot obtain the client information associated with the old node. The file lock cannot be rebuilt on the new node. At the same time, the underlying file system retains the file lock that has been granted on the old node, causing other clients to be permanently unable to obtain the lock, and the storage system cannot provide services to customers normally.

[0074] (3) File lock status recovery must be processed within the grace period of the server, and other lock services cannot be processed during the grace period. When the NFS server is abnormal, it relies on the client to rebuild the file lock. At the same time, the NFS server must enter the grace period, affecting all the basic non-rebuilding lock services of the clients connected to the NFS server.

[0075] Based on the above NFS file lock implementation process, it can be seen that the current file lock service in the distributed storage system has the following pain points: lock consistency is difficult to ensure, and lock status recovery is complicated. In order to solve these technical problems, the present invention provides a file lock implementation solution in a distributed file system. The basic idea of ​​the present invention is:

[0076] The remote procedure call status monitoring service (rpc.statd) and network status monitoring notification service (sm-notify) of the NFS server are merged into the NFS service process to reduce the interaction between processes and improve reliability; when processing the client file lock request, the client information is recorded in the distributed storage system by calling the underlying file system posix interface to solve the problem that the new node of the NFS server cannot obtain the client information of the file lock in the abnormal switching scenario of the NFS server. The technical solution provided by the present invention can effectively ensure the consistency of the file lock in the distributed system, simplify the lock state recovery process, and improve the implementation efficiency and reliability of the file lock.

[0077] The specific implementation process of the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the step processes shown in the accompanying drawings and embodiments can be executed in a computer system such as a set of computer executable instructions, and one or more of the exemplified method steps can be single or multiple or aggregated together and run in the computer system in the form of a software module or a virtual device. Moreover, although the exemplified step processes have time and sequential logic, the present invention is not limited to these steps in certain cases. They can be executed in an order different from that here. Under the guidance of the purpose of the present invention, simple changes or combinations of the execution order of these steps should also fall within the scope of the disclosure of the embodiments of the present invention.

[0078] Figure 6 A schematic flow chart of steps of a method for implementing a distributed file system file lock is provided for one embodiment of the present invention. The method is applied to a network file system server, i.e., an NFS server, in a distributed storage system, and the method comprises:

[0079] S601. After the distributed file system NFS server receives the lock request sent by the NFS client, the internally integrated remote procedure call status monitoring service calls the underlying file system interface to record the client monitoring information in the lock request to the distributed file system;

[0080] S602. The NFS server calls the interface provided by the underlying file system to process the lock request of the NFS client and perform the file lock operation;

[0081] S603. The NFS server feeds back the lock request processing result to the NFS client.

[0082] The specific implementation steps of the distributed file system file lock implementation method provided by the present invention are described in detail below in conjunction with specific embodiments.

[0083] Figure 7 A schematic diagram of the process flow of file lock request and unlock request provided by an embodiment of the present invention. In this embodiment, the NFS server integrates the functions related to file locks of the original rpc.statd process and sm-notify process. The complete process of file lock request processing by the NFS client is as follows:

[0084] S701. The NFS client upper layer application calls the file control function to trigger the execution of the file locking process;

[0085] In the NFS system, the file descriptor can be operated through the file control function (fcntl) function, which provides a variety of functions for controlling the properties of files, such as file locking, blocking, and non-blocking mode settings.

[0086] S702. The NFS client interacts with the local remote procedure call monitoring service (rpc.statd) and persists the client monitoring information locally through the network status monitoring protocol;

[0087] The embodiment of the present invention does not need to modify the NFS client, and keeps the NFS client's processing flow for file locks unchanged. The network lock manager daemon (lockd) of the NFS client kernel communicates with the local remote procedure call monitoring service (rpc.statd) based on the call of fcntl, and creates (creates if it does not exist, opens if it exists) a monitoring information file named NFS server IP through the monitoring (MONITOR) request of the network status monitoring NSM protocol. The file is for the NFS client to record which NFS server holds which file or files' file locks.

[0088] S703. The NFS client sends a lock request to the NFS server based on the remote procedure call (RPC) protocol;

[0089] S704. After the NFS server receives the lock request, the internally integrated remote procedure call status monitoring service (rpc.statd) calls the underlying file system interface to record the client monitoring information in the lock request into the distributed file system;

[0090] After receiving the LOCK request sent by the NFS client through the RPC protocol, the NFS server parses the client monitoring information in the lock request, and then communicates with the remote procedure call status monitoring service (such as rpc.statd) integrated in the NFS service process. Through rpc.statd, the underlying file system interface (such as posix) is called to persistently record the client monitoring information in the lock request to the distributed file system. In addition to the basic client monitoring information (lock type, byte range, etc.), the client's own monitoring information (such as client IP) is also persisted in the distributed file system. The client monitoring information can be stored in the monitoring information file. The monitoring information file is created when processing the first lock of the NFS client and deleted when unlocking the last lock.

[0091] The client monitoring information can be persisted to the metadata server (MDS) of the distributed file system. The client monitoring information in the monitoring information file can be used to notify the NFS client to replay the file lock recorded in the file after the NFS server crashes and recovers, that is, to restore the file lock status based on the client monitoring information.

[0092] Distributed file system (refers to a distributed storage system that supports file storage, such as the Ceph distributed storage system)

[0093] S705. The NFS server calls the interface provided by the underlying file system to process the lock request of the NFS client and perform the file lock operation;

[0094] After recording the client monitoring information in the distributed file system, the NFS server calls the underlying file system to provide an access interface to the distributed file system (such as the posix interface) to perform a locking operation on the files stored in the distributed file system, that is, the locking process is actually performed on the files stored in the distributed file system. The file lock is maintained by the distributed storage system, and the distributed storage system implements mutual exclusive access to the file based on the file lock.

[0095] S706. The NFS server feeds back the lock request processing result to the NFS client.

[0096] After the NFS server completes the file locking process by providing an access interface to the distributed file system through the underlying file system, it returns the processing result of the locking request to the NFS client.

[0097] The following is the file unlocking process:

[0098] S707. The NFS client upper layer application calls the file control function to trigger the execution file unlocking process;

[0099] The file descriptor can be operated by calling a file control function (such as fcntl) interface function to trigger the execution of the file unlocking process.

[0100] S708. The NFS client initiates a file unlock (UNLOCK) request to the NFS server based on the remote procedure call (RPC) protocol;

[0101] S709. After receiving the unlock request from the NFS client, the NFS server calls the distributed file system access interface provided by the underlying file system to unlock the file stored in the distributed file system. After completing the unlock process, the distributed file system returns an unlock success response through the underlying file system interface.

[0102] S710. After the NFS server receives a successful response to the file unlocking, the internally integrated remote procedure call status monitoring service (rpc.statd) calls the interface provided by the underlying file system to delete the client monitoring information stored in the distributed file system.

[0103] After receiving the successful response to file unlocking, the NFS server parses the client information. When it is determined that the file is locked by the NFS client, it calls the interface provided by the underlying file system (such as the posix interface) to delete the client monitoring information of the NFS client stored in the distributed file system, indicating that the NFS client no longer holds the file lock.

[0104] The present invention changes the original inter-process communication between the NFS server and rpc.statd into a function interface call, thereby improving the speed and reliability of unlocking.

[0105] S711. The NFS server feeds back the result of the file unlock request to the NFS client;

[0106] S712. The network lock manager daemon (lockd) of the NFS client kernel communicates with the local remote procedure call monitoring service (rpc.statd) and deletes the monitoring information file named after the NFS server IP through the NSM protocol's UNMONITOR request, indicating that the NFS client no longer holds the file lock for this NFS server.

[0107] By comparing the file locking and unlocking process provided by the embodiment of the present invention and Figure 2 From the file locking and unlocking process, it can be found that due to the integration of rpc.statd in the NFS server, the recording process of the client monitoring information has changed from the original inter-process communication to a more convenient and reliable function interface call. At the same time, the NFS client information monitoring file has a distributed feature and is visible to each node of the distributed storage cluster. In the high-availability switching scenario, each switched NFS server can obtain the client monitoring information in the monitoring file from the distributed file storage system, and has the ability to send NOTIFY to the specified NFS client, thereby improving the efficiency of locking / unlocking and lock replay.

[0108] Figure 8 The figure is a schematic diagram of a file lock replay processing flow triggered by NFS server switching after an NFS server crashes in one embodiment of the present invention.

[0109] When the NFS server loses power or the network is disconnected, in a high-availability scenario, the NFS server will be switched, and the services on the faulty NFS server will be switched to the normal NFS server. As shown in the figure, after NFS client A successfully requests to lock file A through NFS server A (source NFS server), if NFS server A fails and crashes, the distributed file system will perform high-availability switching, and migrate the services provided by NFS server A to NFS client A to NFS server B (the destination NFS server after the switch), and NFS server B will continue to provide services to NFS client A. Usually, NFS server A and NFS server B are located on different storage nodes.

[0110] When the NFS server is switched, NFS server B actively calls the interface provided by the underlying file system to read the client monitoring information file created by NFS server A from the distributed file system, and reads the information of NFS client A from the monitoring file, so that it can accurately send a NOTIFY message to the specified NFS client to notify the NFS client to replay the acquired file lock. In this way, it can effectively solve the problem of high availability scenarios where the new node cannot send a NOTIFY message to the NFS client, resulting in permanent file locks and other NFS clients being unable to obtain locks.

[0111] In a distributed file system, file lock requests are ultimately processed by the MDS (Metadata Service), and the file lock status is ultimately maintained by the MDS, which is the only authority in the distributed file system. Therefore, only when the NFS service node is switched and the MDS does not switch abnormally, the file lock status information of the MDS with only authority is still valid, so no special processing is required.

[0112] Fig. 9 The figure is a schematic diagram of the processing flow of the MDS abnormal switching scenario in one embodiment of the present invention. The file lock request (including lock request / unlock request, etc.) of the NFS client processed by the NFS server is ultimately actually processed and maintained by the MDS, and the MDS will persist the client monitoring information to the storage medium in the distributed storage cluster. The MDS service itself is also deployed in a distributed cluster, and multiple MDS service nodes jointly provide MDS services. The client monitoring information will be persisted in the persistent storage medium of the distributed storage cluster as special data. In the MDS abnormal switching scenario, the switching target MDS can rebuild the client monitoring information and provide file lock services to the upper layer by reading the client monitoring information stored in the persistent storage medium in the distributed storage cluster. Therefore, at the NFS service level, there is no need to rely on the client to rebuild the file lock, and there is no need for the NFS service grace period. The grace period is only briefly entered during the recovery phase after the MDS abnormal switching, which greatly improves the service availability.

[0113] Fig.10 The electronic device structure diagram for implementing the file lock implementation method in the distributed file system provided by the present invention is provided in an embodiment of the present invention. The device 100 includes: a processor 110 such as a central processing unit (CPU), a communication bus 120, a communication interface 140, and a memory 130. The processor 110 and the memory 130 can communicate with each other through the communication bus 120. The memory 130 stores a computer program, and when the computer program is executed by the processor 310, the function of one or more steps in the file lock implementation method in the distributed file system provided by the present invention can be implemented.

[0114] Memory refers to a device based on a certain storage medium for storing computer programs and / or data. It can be a volatile memory (VM, often called memory) or a non-volatile memory (NVM). Memory refers to an internal memory that directly exchanges data with the processor. It can read and write data at any time and at a very fast speed. It is used as a storage medium for temporary data of the operating system and other running programs. Memory can be synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), etc. Non-volatile memory refers to a memory that uses a persistent storage medium. It has the characteristics of large capacity and persistent data storage. It can be a storage class memory (SCM), a solid state disk (SSD), a NAND flash memory, a disk, etc. SCM is the industry's general term for new storage media between memory and flash memory. It is a composite storage technology that combines persistent storage characteristics and memory characteristics. Its access speed is slower than DRAM and faster than SSD hard disks.

[0115] The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0116] It should be appreciated that embodiments of the present invention may be implemented or implemented by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory (or non-persistent) memory. The method may be implemented in a computer program using standard programming techniques, including a non-transitory storage medium configured with a computer program, wherein the storage medium configured in this way causes the computer to operate in a specific and predefined manner. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if necessary, the program may be implemented in an assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed ASIC for this purpose. In addition, the operation of the process described in the present invention may be performed in any suitable order, unless the present invention otherwise indicates or otherwise clearly contradicts the context. The process described in the present invention (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as a code (e.g., executable instructions, one or more computer programs, or one or more applications) executed on one or more processors in common, by hardware or a combination thereof. The computer program includes a plurality of instructions that may be executed by one or more processors.

[0117] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, a RAM, a ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or portions thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described in the present invention includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself.

[0118] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for implementing file lock in a distributed file system, characterized in that: The method includes: After the distributed file system NFS server receives the lock request sent by the NFS client, the internally integrated remote procedure call status monitoring service calls the underlying file system interface to record the client monitoring information in the lock request into the distributed file system; The NFS server calls the interface provided by the underlying file system to process the lock request of the NFS client and perform the file lock operation; The NFS server feeds back the lock request processing result to the NFS client.

2. The method according to claim 1, characterized in that The method further comprises: After receiving the unlock request from the NFS client, the NFS server calls the interface provided by the underlying file system to unlock the file stored in the distributed file system; After the NFS server receives the file unlock success response, if it is determined that the file is locked by the NFS client for the last time, the internally integrated remote procedure call status monitoring service calls the interface provided by the underlying file system to delete the client monitoring information stored in the distributed file system; The NFS server feeds back the result of the file unlock request to the NFS client.

3. The method according to claim 2, characterized in that The method further comprises: In the scenario where NFS server switching occurs, the destination NFS server takes over the services of the source destination NFS server; When the NFS server switch is completed, the destination NFS server calls the interface provided by the underlying file system to read the client monitoring information created by the source NFS server from the distributed file system; The destination NFS server sends a NOTIFY message to the NFS client based on the read client monitoring information, so that the NFS client replays the acquired file lock.

4. The method according to claim 3, characterized in that The method further comprises: The client monitoring information is stored in the metadata service MDS in the distributed file system; In the MDS abnormal switching scenario, the switching target MDS reads the client monitoring information stored in the persistent storage medium in the distributed storage cluster, rebuilds the client monitoring information and provides file lock services for the upper layer.

5. The method according to claim 1, characterized in that The NFS server integrates not only the remote procedure call monitoring service (rpc.statd) process, but also the network status monitoring notification service (sm-notify) process.

6. A file lock implementation device in a distributed file system, characterized in that: The device is applied to a distributed file system NFS server, and includes: The file lock monitoring information processing module is used to, after receiving the lock request sent by the NFS client, call the underlying file system interface by the internally integrated remote procedure call status monitoring service to record the client monitoring information in the lock request into the distributed file system; The file lock execution module is used to call the interface provided by the underlying file system, process the lock request of the NFS client, and execute the file lock operation; The lock request response module is used to feedback the lock request processing result to the NFS client.

7. The device according to claim 6, characterized in that The file lock execution module is further used to call the interface provided by the underlying file system to perform an unlocking operation on the file stored in the distributed file system after receiving an unlocking request from the NFS client; The file lock monitoring information processing module is further used to, after receiving a successful file unlock response, if it is determined that the file is the last file lock of the NFS client, call the interface provided by the underlying file system through the internally integrated remote procedure call status monitoring service to delete the client monitoring information stored in the distributed file system; The lock request response module is also used to feed back the file unlock request processing result to the NFS client.

8. The device according to claim 7, characterized in that In the scenario where NFS server switching occurs, the destination NFS server takes over the services of the source destination NFS server; The file lock monitoring information processing module is also used to call the interface provided by the underlying file system to read the client monitoring information created by the source NFS server from the distributed file system after the NFS server switching is completed; The file lock monitoring information processing module is further used to send a NOTIFY message to the NFS client based on the read client monitoring information, so that the NFS client replays the acquired file lock.

9. The device according to claim 6, characterized in that The NFS server integrates not only the remote procedure call monitoring service (rpc.statd) process, but also the network status monitoring notification service (sm-notify) process.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a storage medium and a communication bus, wherein the processor, the communication interface and the storage medium communicate with each other via the communication bus; Storage medium for storing computer programs; A processor, for implementing the method according to any one of claims 1 to 5 when executing a computer program stored on a storage medium.

Images