Method and device for processing NFS (Network File System) service fault in distributed file system

Abstract

The embodiment of the invention provides a processing method and device for an NFS service fault in a distributed file system. In the application, when the NFS server side in the distributed file system executes the file lock operation on the file every time, the corresponding file lock information is persistently stored in the extended attribute of the file for recording, so that the file lock of each NFS server side in the distributed file system has sharing performance; different NFS servers can share the file lock by accessing the record persistently stored in the extended attribute of the file, which is helpful for ensuring the consistency of the file lock during service failover. Whether each NFS server side of the whole system has a service fault is monitored from a global perspective through a main monitoring node and a slave monitoring node in the distributed file system, so that file lock recovery between the NFS server side and each client side corresponding to a fault NFS service is coordinated and replaced during service fault switching; therefore, the consistency of file locks between the substitute NFS server and each client is ensured.

Description

Technical Field

[0001] The present application relates to the field of distributed storage technology, and in particular to a method and device for handling NFS service failures in a distributed file system. Background Art

[0002] Distributed file systems are widely used due to their scalability, high availability, and high performance. In distributed file systems, file locks are needed to coordinate access to the same file by multiple Network File System (NFS) clients to ensure data consistency.

[0003] For multiple NFS servers of a distributed file system, each NFS server maintains its own file lock. That is, each NFS server stores its own file lock in its memory, and other NFS servers do not share the file lock in the NFS server memory.

[0004] However, when an NFS server fails, the client (also called the NFS client) automatically establishes a connection with another NFS server and continues to use the previous file lock to access the previous file. However, the newly connected NFS server cannot obtain the file lock in the memory of the failed NFS server, which leads to inconsistent file locks between the server and the client. In other words, it is difficult to ensure file lock consistency during service failover in a distributed file system. Summary of the Invention

[0005] In view of this, embodiments of the present application provide a method and apparatus for handling NFS service failures in a distributed file system, so as to ensure the consistency of file locks during service failure switching.

[0006] The present invention provides a method for handling NFS service failures in a distributed file system. The method is applied to an NFS server in the distributed file system. The method includes:

[0007] Each time a file lock operation is performed on a file, the corresponding file lock information is persistently stored in the file's extended attributes.

[0008] When the service node in the present NFS server is the master monitoring node in the distributed file system, if the master monitoring node monitors that an NFS service provided by an NFS server has failed, the master monitoring node determines an alternative NFS server from the distributed file system and migrates the virtual IP address corresponding to the failed NFS service to the alternative NFS server, so that the alternative NFS server establishes a network connection with each client corresponding to the failed NFS service based on the migrated virtual IP address, and the master monitoring node sends a service recovery notification to each client based on the network connection, thereby triggering each client to send a file lock recovery request to the virtual IP address;

[0009] When the service node in the NFS server is a slave monitoring node in the distributed file system, it monitors whether the NFS service provided by the NFS server has any failure and notifies the master monitoring node;

[0010] When receiving a file lock recovery request sent by the client, it determines whether the file lock recovery request is legal based on the file lock information recorded in the file extended attributes corresponding to the faulty NFS service, and triggers the client to continue using the original file lock on this NFS server if the file lock recovery request is legal.

[0011] The present application also provides a device for processing an NFS service failure in a distributed file system. The device is applied to an NFS server in the distributed file system. The device includes:

[0012] The persistence module is used to store the corresponding file lock information persistently in the file's extended attributes each time a file lock operation is performed on the file;

[0013] A sending module is configured to, when the service node in the present NFS server is a primary monitoring node in the distributed file system, determine, through the primary monitoring node, an alternative NFS server from the distributed file system if an NFS service provided by the NFS server is detected to have failed through the primary monitoring node, and migrate the virtual IP address corresponding to the failed NFS service to the alternative NFS server, so that the alternative NFS server can establish a network connection with each client corresponding to the failed NFS service based on the migrated virtual IP address, and send a service recovery notification to each client through the primary monitoring node based on the network connection, thereby triggering each client to send a file lock recovery request to the virtual IP address;

[0014] When the service node in the NFS server is a slave monitoring node in the distributed file system, it monitors whether the NFS service provided by the NFS server has any failure and notifies the master monitoring node;

[0015] The determination module is used to determine whether the file lock recovery request sent by the client is legal based on the file lock information recorded in the file extension attribute corresponding to the faulty NFS service when the file lock recovery request is received, and trigger the client to continue using the original file lock on the current NFS server if the file lock recovery request is legal.

[0016] An embodiment of the present application further provides an electronic device, comprising: a processor and a computer-readable storage medium for storing computer program instructions, wherein the computer program instructions, when executed by the computer-readable storage medium, enable the processor to execute the steps of the above method.

[0017] An embodiment of the present application further provides a machine-readable storage medium, which stores computer program instructions. When the computer program instructions are executed, the steps of the above method can be implemented.

[0018] It can be seen from the above technical solution that in this embodiment, each time the NFS server in the distributed file system performs a file lock operation on a file, the corresponding file lock information is persistently stored in the extended attributes of the file, so that the file locks of each NFS server in the distributed file system are shared. Different NFS servers can share file locks by accessing the records persistently stored in the extended attributes of the file, which helps to ensure the consistency of file locks during service failover.

[0019] Furthermore, NFS service failures are monitored via the master and slave monitoring nodes in the distributed file system. When a failure in an NFS service provided by an NFS server is detected, the master monitoring node determines a replacement NFS server from the distributed file system and migrates the virtual IP address corresponding to the failed NFS service to the replacement NFS server. The replacement NFS server then establishes a network connection with each client corresponding to the failed NFS service based on the migrated virtual IP address. The master monitoring node then sends a service recovery notification to each client, triggering each client to send a file lock recovery request to the virtual IP address. This allows the master and slave monitoring nodes in the distributed file system to globally monitor each NFS server in the entire system for service failures. This allows file lock recovery between the replacement NFS server and each client corresponding to the failed NFS service to be coordinated during a service failover, thereby ensuring file lock consistency between the replacement NFS server and each client. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic flow chart of the method provided in the embodiment of the present application;

[0021] Figure 2 A schematic diagram of a process for clearing a file lock cache provided in an embodiment of the present application;

[0022] Figure 3 A schematic flow chart of the method provided in the embodiment of the present application;

[0023] Figure 4 A schematic diagram of the structure of the device provided in the embodiment of the present application;

[0024] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0026] It should be noted that the data analysis method provided in the embodiments of the present application is applicable to data analysis of data in databases in the medical field, and is also applicable to data analysis of data in databases in other fields such as the financial field.

[0027] See also Figure 1 , Figure 1 The method flow diagram provided in the embodiment of the present application is as follows: As an embodiment, the method is executed by an NFS server in a distributed file system.

[0028] like Figure 1 As shown, the process may include the following steps:

[0029] S101, each time a file lock operation is performed on a file, corresponding file lock information is persistently stored in an extended attribute of the file.

[0030] In this embodiment, the file lock nlm_lock operation includes: a locking operation and an unlocking operation.

[0031] The file lock information includes at least: the data range in the file corresponding to the file lock (the data range that the client needs to process can be represented by the data offset offset and the data length len), the type of the file lock operation type (here, the type includes shared lock and exclusive lock), the state of the file lock state (here, the state includes granted and released), the host identifier of the file lock requester, the process identifier of the file lock requester, the file lock owner identifier, and the file identifier of the file on which the file lock operation is performed.

[0032] For example, in a Linux distributed file system, the host ID of the file lock requester, the process ID of the file lock requester, and the file lock owner ID are represented by three fields: {caller, owner_handle, svid}.

[0033] Among them, the host identifier caller of the file lock requester is a pointer to nlm_host, which is represented by nlm_host{host_addr, host_name, host_state}, where host_addr is the host's IP address, host_name is the host name, and host_state is the host state.

[0034] The file lock owner identifier owner_handle is a unique identifier, generated by the requester and unique during the life cycle of the lock.

[0035] The process identifier svid of the file lock requester is the process ID on the requester's host.

[0036] The file identifier on which the file lock operation is performed is generally represented by the super block (super_block) of the distributed file system and the unique identifier of the file metadata inode.

[0037] In other words, file lock information, namely offset, len, type, state, caller, owner_handle, svid, etc., is saved in the extended attributes of the corresponding file in the form of key-value pairs.

[0038] For example, the file lock information persistently stored in the extended attributes of a file is “nfs.file.nlm_lock”: “{.offset=xxx,.len=xxx,.type=xxx,.state=xxx,.caller=xxx,.owner_handle=xxx,.svid=xxx}”.

[0039] Among them, "nfs.file.nlm_lock" is used as the key of the extended attribute xattr, and "{.offset=xxx,.len=xxx,.type=xxx,.state=xxx,.caller=xxx,.owner_handle=xxx,.svid=xxx}" is used as the value of the extended attribute xattr, both of which are stored in string format.

[0040] In this step, the distributed file system already supports extended attributes of file metadata, eliminating the need to modify the original metadata structure. Furthermore, the file's extended attributes are persisted to the physical storage medium via the metadata server, ensuring data redundancy and reliability. The file locks of each NFS server in the distributed file system are shared. For example, NFS server A can obtain file lock information (i.e., the file lock content and status described above) by accessing the file lock information within the extended attributes of the file corresponding to NFS server B, thereby ensuring the reliability and consistency of file locks during NFS service failover.

[0041] The above step S101 is described in detail.

[0042] S102, when the service node in this NFS server is the main monitoring node in the distributed file system, if the main monitoring node monitors that an NFS service provided by an NFS server has failed, the main monitoring node determines an alternative NFS server from the distributed file system, and migrates the virtual IP address corresponding to the failed NFS service to the alternative NFS server, so that the alternative NFS server establishes a network connection with each client corresponding to the failed NFS service based on the migrated virtual IP address, and sends a service recovery notification to each client based on the network connection through the main monitoring node, so as to trigger each client to send a file lock recovery request to the virtual IP address.

[0043] S103, when the service node in the NFS server is a slave monitoring node in the distributed file system, monitor whether the NFS service provided by the NFS server fails, and notify the master monitoring node.

[0044] In this embodiment, each service node is configured with a monitoring service monitor, wherein one monitor is designated as the master monitor (the service node where the slave monitor is located is the master monitoring node), and the other monitors are slave monitors (the service node where the slave monitor is located is the slave monitoring node).

[0045] Each slave monitoring node monitors whether the NFS service provided by the NFS server deployed thereon has failed. As an embodiment, the specific implementation method of monitoring whether the NFS service has failed may be: within a specified test time period, if the delay time of the Ping test instruction sent based on the virtual IP address corresponding to the NFS service exceeds a set threshold (that is, the virtual IP address corresponding to the NFS service is unreachable), then it is determined that the NFS service has failed. If the slave monitoring node monitors that the NFS service provided by the NFS server deployed thereon has failed, it notifies the master monitoring node. The master monitoring node monitors whether the NFS service provided by the NFS server deployed thereon has failed.

[0046] The slave monitoring node will report the status of the service node to the master monitoring node multiple times within one second, and a type of NFS service failure / recovery event will be attached to the reported status. When the master monitoring node obtains that an NFS service provided by an NFS server has failed, it will determine an alternative NFS server from the distributed file system based on the configured high availability group information and load balancing strategy (for example, according to the load balancing principles specified by CPU load / network bandwidth / number of connections / polling, etc.). The master monitoring node sends a notification to the NFS server where the failed NFS service occurs (for ease of description, it is recorded as the original NFS server) and the slave monitoring node where the selected alternative NFS server is located, and specifies that the slave monitoring node executes the migration of the virtual IP address corresponding to the failed NFS service to the alternative NFS server. Optionally, as an embodiment, the virtual IP address on the NFS port of the service node where the NFS server where the failed NFS service occurs is located is deleted, and the virtual IP address is set on the NFS port of the service node where the alternative NFS server is located. In this way, each client corresponding to the failed NFS service establishes a network connection with the replacement NFS server based on the migrated virtual IP address. That is, each client corresponding to the failed NFS service is unaware that the previous NFS service has failed and still uses the virtual IP address for access. At this time, the replacement NFS server provides services to each client.

[0047] After the replacement NFS server establishes a network connection with each client corresponding to the failed NFS service, the master monitoring node obtains the IP address of each client connected to the virtual IP address corresponding to the failed NFS service (it is known that in a distributed file system, when a client establishes a network connection with an NFS server, the slave monitoring node corresponding to the NFS server will report the client's IP address to the master monitoring node, and the master monitoring node will record the IP address). The master monitoring node sends the IP address of each client connected to the virtual IP address to the slave monitoring node corresponding to the replacement NFS server. The slave monitoring node sends a service recovery notification to each client based on the IP address of each client through the statd module. The nlm_host content in the service recovery notification is the identifier of the failed NFS service. In this way, each client corresponding to the failed NFS service knows that the NFS service has returned to normal, and each client sends a file lock recovery request to the virtual IP address.

[0048] The above steps S102 and S103 are described in detail.

[0049] S104, when receiving the file lock recovery request sent by the client, determine whether the file lock recovery request is legal based on the file lock information recorded in the file extension attribute corresponding to the failed NFS service, and trigger the client to continue using the original file lock on the current NFS server if the file lock recovery request is legal.

[0050] In this step, only when this NFS server is determined as the replacement NFS server will it receive file lock recovery requests sent by the clients corresponding to the failed NFS service. Previously, the master monitoring node determined the replacement NFS server. After the master monitoring node determined the replacement NFS server, it sent a query notification to the slave monitoring node where the replacement NFS server is located, and the slave monitoring node sent the query notification to the replacement NFS server. The replacement NFS server finds the file lock information recorded in the file extended attribute corresponding to the failed NFS service from the extended attributes of the file metadata of the shared distributed file system based on the failed NFS service.

[0051] When the NFS server receives a file lock recovery request from the client, it determines whether the file lock recovery request is valid based on the file lock information recorded in the file extended attributes corresponding to the faulty NFS service, and triggers the client to continue using the original file lock on the NFS server if the file lock recovery request is valid.

[0052] As an embodiment, a specific implementation method for determining a file lock recovery request based on the file lock information recorded in the file extended attribute corresponding to the faulty NFS service may include: determining whether the file lock recovery request information matches the target file lock information in the file lock information recorded in the file extended attribute corresponding to the faulty NFS service; if so, determining that the file lock recovery request is valid; if not, determining that the file lock recovery request is invalid. The target file lock information refers to the file lock information that matches the host ID of the file lock requester, the process ID of the file lock requester, the file lock owner ID, the file ID, the file lock type, and the file lock status carried in the file lock recovery request.

[0053] The above step S104 is described in detail.

[0054] In steps S102, S103, and S104, the distributed file system's existing monitoring service coordinates the NFS server and client to restore file locks and achieve rapid file lock service switching. The master and slave monitoring nodes in the distributed file system directly monitor the NFS service status. In addition to monitoring NFS service status changes such as failures and recovery, they also coordinate lock recovery actions on the client.

[0055] It should be noted that the related art uses statd and lockd to work together to restore file locks previously held by the client. This approach only supports one-to-one heartbeat monitoring between the client and server. File lock recovery is only possible when the client and server are both down and restarted. This approach is not applicable to service failover in distributed file systems. Furthermore, in situations where this approach is applicable, to prevent confusion, for a period of time after service restoration (i.e., the NFS grace period), the server only processes file lock recovery requests sent by the client and does not process new lock requests. Obviously, when the service is restored, the grace period of the NFS service is too long, making it impossible to continue providing NFS lock services in the short term. Distributed file systems have high requirements for business continuity. However, after the client establishes a network connection with the replacement NFS server, if the NFS grace period mechanism is directly used, the NFS service will take 60 seconds or even longer to resume providing file lock services. In other words, an excessively long NFS grace period affects system efficiency.

[0056] In the above-mentioned method of the present application, since common distributed file systems have a monitoring service, the monitoring service can quickly detect changes in the file system status, including NFS service failures and recovery. By directly utilizing the existing monitoring service of the distributed file system and leveraging its fast response characteristics, the NFS lock service can be quickly switched, effectively avoiding the long switching overhead of the NFS grace period in related technologies.

[0057] Furthermore, whether it is persistent storage of file lock information or use of monitoring services, it is done by relying on the original network structure of the distributed file system, and does not change the original structure of metadata storage, which can improve the applicability of the method provided in the embodiment of the present application.

[0058] So far, completed Figure 1 The process shown.

[0059] pass Figure 1 As can be seen from the illustrated process, in this embodiment, each time the NFS server in the distributed file system performs a file lock operation on a file, the corresponding file lock information is persistently stored in the extended attributes of the file, so that the file locks of each NFS server in the distributed file system are shared. Different NFS servers can share the file locks by accessing the records persistently stored in the extended attributes of the file, which helps to ensure the consistency of the file locks during service failover.

[0060] Furthermore, NFS service failures are monitored via the master and slave monitoring nodes in the distributed file system. When a failure in an NFS service provided by an NFS server is detected, the master monitoring node determines a replacement NFS server from the distributed file system and migrates the virtual IP address corresponding to the failed NFS service to the replacement NFS server. The replacement NFS server then establishes a network connection with each client corresponding to the failed NFS service based on the migrated virtual IP address. The master monitoring node then sends a service recovery notification to each client, triggering each client to send a file lock recovery request to the virtual IP address. This allows the master and slave monitoring nodes in the distributed file system to globally monitor each NFS server in the entire system for service failures. This allows file lock recovery between the replacement NFS server and each client corresponding to the failed NFS service to be coordinated during a service failover, thereby ensuring file lock consistency between the replacement NFS server and each client.

[0061] The cause of the above-mentioned NFS service failure may not only be that the entire server is down (the entire server can be restored by re-uploading the file lock recovery process in the relevant technology, which will not be repeated here), but also that part of the network of the NFS server is abnormal, for example, the network port of the virtual IP address set by the NFS server is abnormal), and the virtual IP address corresponding to the faulty port of the NFS server drifts to other NFS servers, but other network ports other than the faulty network port can continue to provide services (that is, other virtual IP addresses maintain the original binding). In order to prevent clients using other virtual IP addresses from accessing the old cache of the file lock corresponding to the faulty service in the original NFS, it is necessary to clean up the file lock cache corresponding to the virtual IP address corresponding to the faulty NFS service. Moreover, cleaning the file lock cache must be performed before the client sends a request to restore the file lock to the alternative NFS server. Only in this way can the consistency of the lock cache be guaranteed.

[0062] It can be seen from this that in a distributed file system, when an NFS service fails, before the client establishes a network connection with the replacement NFS server, in order to ensure the consistency of file locks within the system, it is necessary to ensure that the file locks corresponding to the failed NFS in the memory of the NFS server where the failed NFS service occurred (for the sake of convenience, it is referred to as the original NFS server) have been cleared.

[0063] Therefore, after the master monitoring node determines the replacement NFS server, the master monitoring node sends a cache clearing notification to the NFS server of the failed NFS service, so that the NFS server of the failed NFS service clears the file lock cache corresponding to the failed NFS service in its local memory. As an embodiment, the master monitoring node sends a cache clearing notification to the slave monitoring node where the original NFS server is located. The slave monitoring node sends the cache clearing notification to the original NFS server. After receiving the cache clearing notification, the original NFS server clears the file lock cache corresponding to the failed NFS service stored in its local memory.

[0064] Considering that file lock cleanup is very time-consuming and becomes a performance bottleneck for NFS service switching, the following method is used to clean up the file lock cache.

[0065] See also Figure 2 , Figure 2 A schematic diagram of the process of clearing the file lock cache provided in an embodiment of the present application.

[0066] like Figure 2 As shown, the process may include the following steps:

[0067] S201: Increase the version number of the file lock queue corresponding to the virtual IP address corresponding to the faulty NFS service stored in the local memory by a set value, so that the state of the file lock cache in the file lock queue existing before receiving the cache clearing notification is set to an expired state.

[0068] In this embodiment, the NFS server is configured with at least one virtual IP address. Each virtual IP address serves different clients and applications. When the NFS service corresponding to a certain virtual IP address fails and needs to be migrated to another NFS server, it is necessary to ensure that the file lock cache related to the virtual IP address is cleared to avoid inconsistent lock caches. The file lock caches of different virtual IP addresses need to be isolated, and migrating one virtual IP address cannot affect the lock status of the file locks of other virtual IP addresses. Therefore, the file lock caches corresponding to different virtual IP addresses are recorded in different file lock queues. Specifically, the file lock queues are divided into multiple hash buckets, and the file lock information of different virtual IP addresses is stored in different file lock queues through hash operations. In this way, when the virtual IP address corresponding to the failed NFS service is deleted, the index to the file lock queue corresponding to the failed NFS service can be accelerated.

[0069] In order to reduce the time for clearing the waiting cache, the version number of the file lock queue corresponding to the virtual IP address corresponding to the faulty NFS service stored in the local memory is increased by a set value, so that the status of the file lock cache in the file lock queue that existed before receiving the cache clearing notification is set to expired.

[0070] Specifically, the present application adds a version number to the file lock queue. The version number is globally unique after the NFS service is started, and is used after the set value is incremented (for example, plus 1) when needed. When the NFS service fails and the corresponding virtual IP address drifts to other clients, the version of the file lock queue corresponding to the virtual IP address increases by 1, but the version number of the file lock in the file lock queue that existed before receiving the cache clearing notification is not increased. For example, the version of the file lock queue increases by 1 to version 2, while the version number of the file lock in the file lock queue that existed before receiving the cache clearing notification is version 1.

[0071] At this point, subsequent file lock replies and newly created file locks in the client are based on the latest version number. The status of the file lock cache in the file lock queue that existed before receiving the cache clear notification is set to expired.

[0072] If the client accesses the file lock through another virtual IP address, since the file lock version is lower than the latest file lock queue version, the file lock is clearly expired, the expired file lock cache is ignored, and the latest file lock is obtained from the metadata extended attributes.

[0073] If the original NFS server is restored, it will not provide services according to the old file lock cache. The lock version requested by the client matches the latest lock queue version.

[0074] S202: Clearing the expired file lock cache at specified time intervals.

[0075] So far, completed Figure 2 The process shown.

[0076] pass Figure 2 As can be seen from the illustrated process, this embodiment accelerates the clearing of the NFS server's file lock cache through lock queue version management. This method achieves the fastest file lock cache clearing process, significantly shortening the NFS service switching time. This overcomes the limitations of the NFS grace period mechanism, enabling rapid NFS service recovery and switching within seconds, eliminating the performance bottleneck caused by file lock cache clearing during NFS switching.

[0077] As an example, after the replacement client determines that the file lock recovery request is legitimate, it sends a file lock recovery success message to the client that sent the file lock recovery request. Based on the file lock recovery success message, the client updates its local file lock information, updates the status of the obtained file lock to granted, records it in the local lock queue, and performs local processing. The client can then continue to use the file lock on the replacement NFS server.

[0078] In order to explain the technical solution provided by the present application in more detail, the technical solution provided by the present application is explained in detail below by way of specific embodiments:

[0079] See also Figure 3 , Figure 3 This is a flow chart of the method provided in the embodiment of the present application. Figure 3 As shown, the process includes the following steps:

[0080] S301 , each NFS server in the distributed storage system stores the corresponding file lock information persistently in the extended attributes of the file each time a file lock operation is performed on the file.

[0081] S302: When the slave monitoring node A detects that the NFS service provided by the NFS server A fails, the slave monitoring node A reports the NFS service failure to the master monitoring node M.

[0082] In this step, NFS server A is deployed on service node A (ie, slave monitoring node A).

[0083] S303, the main monitoring node M obtains the report message of the NFS service failure, and determines the NFS server T as the replacement NFS server (denoted as replacement NFS server T) from the distributed file system based on the configured high availability group information and load balancing strategy.

[0084] In this step, the replacement NFS server T is deployed on the service node T (ie, the slave monitoring node T).

[0085] S304, the master monitoring node M sends a notification to the slave monitoring node A and the slave monitoring node T. The slave monitoring node A sets the virtual IP address on the NFS port of the NFS server A, and the slave monitoring node T sets the virtual IP address on the NFS port of the replacement NFS server T. Each client corresponding to the failed NFS service establishes a network connection with the replacement NFS server T based on the migrated virtual IP address.

[0086] S305, the master monitoring node M sends a cache clearing notification to the slave monitoring node A, and the monitoring node A sends the cache clearing notification to the NFS server A where the NFS service fails. After receiving the cache clearing notification, the NFS server A clears the file lock cache corresponding to the failed NFS service stored in the local memory.

[0087] The specific implementation of step S305 is detailed in the above Figure 2 The embodiments shown will not be described in detail here.

[0088] S306, the master monitoring node M sends a query notification to the slave monitoring node T, and the slave monitoring node T sends the query notification to the alternative NFS server T. The alternative NFS server T finds the file lock information of the file extended attribute record corresponding to the faulty NFS service from the extended attributes of the file metadata of the shared distributed file system.

[0089] S307, the main monitoring node M obtains the IP address of each client connected to the virtual IP address corresponding to the faulty NFS service, and the main monitoring node M sends the obtained IP address of each client to the slave monitoring node T. The slave monitoring node T sends a service recovery notification to each client based on the IP address of each client through the statd module. The nlm_host content in the service recovery notification is the identifier of the faulty NFS service.

[0090] In this way, each client corresponding to the faulty NFS service knows that the NFS service has returned to normal, and each client sends a file lock recovery request to the virtual IP address.

[0091] S308 , after receiving the service recovery notification through the statd module, each client sends a file lock recovery request to the alternative NFS server T through the virtual IP address.

[0092] S309: When the replacement NFS server T receives the file lock recovery request sent by the client, it determines whether the file lock recovery request is legal based on the file lock information of the file extended attribute record corresponding to the failed NFS service found previously.

[0093] The specific implementation of step S309 refers to the specific description of step S104 above, which will not be repeated here.

[0094] S3010: After determining that the file lock recovery request is valid, the alternative NFS server T sends a file lock successful recovery message to the client that sent the file lock recovery request.

[0095] S3011: The client updates the local file lock information based on the file lock successful recovery message. The client updates the status of the obtained file lock to granted, records it in the local lock queue, and performs local processing. The client can continue to use the previous file lock on the alternative NFS server.

[0096] The above describes the method provided in the embodiment of the present application. The following describes the device provided in the embodiment of the present application:

[0097] See also Figure 3 , Figure 3 This is a diagram of the device structure provided in the embodiment of this application. Figure 3As shown, the device is applied to an NFS server in a distributed file system, and the device includes: a persistence module 401, a sending module 402, and a determination module 403.

[0098] The persistence module 401 is used to persistently store the corresponding file lock information in the extended attributes of the file each time a file lock operation is performed on the file;

[0099] A sending module 402 is configured to, when a service node in the present NFS server is a primary monitoring node in the distributed file system, determine, through the primary monitoring node, an alternative NFS server from the distributed file system if an NFS service provided by the NFS server is detected to have failed by the primary monitoring node, and migrate the virtual IP address corresponding to the failed NFS service to the alternative NFS server, so that the alternative NFS server establishes a network connection with each client corresponding to the failed NFS service based on the migrated virtual IP address, and send, through the primary monitoring node, a service recovery notification to each client based on the network connection, thereby triggering each client to send a file lock recovery request to the virtual IP address;

[0100] When the service node in the NFS server is a slave monitoring node in the distributed file system, monitor whether the NFS service provided by the NFS server fails and notify the master monitoring node;

[0101] Determination module 403 is used to determine whether the file lock recovery request sent by the client is legal based on the file lock information recorded in the file extension attribute corresponding to the faulty NFS service when the file lock recovery request is received, and trigger the client to continue using the original file lock on the current NFS server if the file lock recovery request is legal.

[0102] As an embodiment, the file lock operation includes: a locking operation and an unlocking operation;

[0103] The file lock information includes at least: the data range in the file corresponding to the file lock, the type of the file lock, the status of the file lock, the host identifier of the file lock requester, the process identifier of the file lock requester, the file lock owner identifier, and the file identifier of the file on which the file lock is executed.

[0104] As an embodiment, after determining the alternative NFS server, the sending module is further configured to: send a query notification to the alternative NFS server through the primary monitoring node, so that the alternative NFS server finds file lock information of a file extended attribute record corresponding to the faulty NFS service based on the faulty NFS service;

[0105] When the service node in the NFS server is a slave monitoring node in the distributed file system, the device further includes:

[0106] The query module is further used to, when receiving the query notification, find the file lock information of the file extension attribute record corresponding to the faulty NFS service based on the query notification, so that when the NFS server receives the file lock recovery request sent by each client corresponding to the faulty NFS service, it can determine whether the file lock recovery request is legal based on the found file lock information.

[0107] As an embodiment, after determining the replacement NFS server, the sending module is further configured to: send a cache clearing notification to the NFS server where the faulty NFS service occurs through the primary monitoring node, so that the NFS server where the faulty NFS service occurs clears the file lock cache corresponding to the faulty NFS service in the local memory;

[0108] When the service node in the NFS server is a slave monitoring node in the distributed file system, the device further includes:

[0109] The cache cleaning module is used to clear the file lock cache corresponding to the faulty NFS service stored in the local memory when receiving the cache clearing notification.

[0110] As an embodiment, the NFS server is configured with at least one virtual IP address; file lock cache records corresponding to different virtual IP addresses are in different file lock queues;

[0111] The clearing of the file lock cache corresponding to the faulty NFS service stored in the local memory includes:

[0112] Increasing the version number of the file lock queue corresponding to the virtual IP address corresponding to the faulty NFS service stored in the local memory by a set value, so that the state of the file lock cache in the file lock queue existing before receiving the cache clearing notification is set to an expired state;

[0113] The file lock cache in the expired state is cleared at intervals of a specified time period.

[0114] As an embodiment, determining whether the file lock recovery request is legal based on the file lock information recorded by the file extension attribute corresponding to the failed NFS service includes:

[0115] If the file lock information of the file extended attribute record corresponding to the faulty NFS service contains target file lock information that matches the file lock recovery request information, then the file lock recovery request is determined to be legal;

[0116] The target file lock information refers to: the file lock information that matches the host identifier of the file lock requester, the process identifier of the file lock requester, the file lock owner identifier, the file identifier, the file lock type, and the file lock status carried in the file lock recovery request;

[0117] If the target file lock information matching the file lock recovery request information does not exist in the file lock information of the file extended attribute record corresponding to the faulty NFS service, it is determined that the file lock recovery request is illegal.

[0118] As an embodiment, after determining that the file lock recovery request is legal, the sending module is further configured to:

[0119] A file lock success recovery message is sent to the client that sent the file lock recovery request, so that the client updates local file lock information based on the file lock success recovery message and continues to use the recovered file lock.

[0120] So far, completed Figure 4 Structural description of the device shown.

[0121] See also Figure 5 , Figure 5 This is a structural diagram of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0122] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0123] Exemplarily, the machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or similar storage media, or a combination thereof.

[0124] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for handling NFS service failure in a distributed file system, characterized in that: The method is applied to an NFS server in a distributed file system, and the method includes: Each time a file lock operation is performed on a file, the corresponding file lock information is persistently stored in the extended attributes of the file; When the service node in the present NFS server is a master monitoring node in the distributed file system, if the master monitoring node monitors that an NFS service provided by an NFS server has failed, the master monitoring node determines an alternative NFS server from the distributed file system, and migrates the virtual IP address corresponding to the failed NFS service to the alternative NFS server, so that the alternative NFS server establishes a network connection with each client corresponding to the failed NFS service based on the migrated virtual IP address, and the master monitoring node sends a service recovery notification to each client based on the network connection, thereby triggering each client to send a file lock recovery request to the virtual IP address; When the service node in the NFS server is a slave monitoring node in the distributed file system, monitor whether the NFS service provided by the NFS server fails and notify the master monitoring node; When receiving a file lock recovery request sent by the client, it determines whether the file lock recovery request is legal based on the file lock information recorded in the file extended attributes corresponding to the faulty NFS service, and triggers the client to continue using the original file lock on this NFS server if the file lock recovery request is legal.

2. The method according to claim 1, characterized in that The file lock operation includes: locking operation and unlocking operation; The file lock information includes at least: the data range in the file corresponding to the file lock, the type of the file lock, the status of the file lock, the host identifier of the file lock requester, the process identifier of the file lock requester, the file lock owner identifier, and the file identifier of the file on which the file lock is executed.

3. The method according to claim 1, characterized in that After determining the alternative NFS server, the method further includes: Sending a query notification to the substitute NFS server through the primary monitoring node, so that the substitute NFS server finds file lock information of the file extended attribute record corresponding to the faulty NFS service based on the faulty NFS service; When the service node in the NFS server is a slave monitoring node in the distributed file system, the method further includes: When the query notification is received, the file lock information of the file extension attribute record corresponding to the faulty NFS service is found based on the query notification, so that when the NFS server receives the file lock recovery request sent by each client corresponding to the faulty NFS service, it can determine whether the file lock recovery request is legal based on the found file lock information.

4. The method according to claim 1, wherein After determining the alternative NFS server, the method further includes: Sending a cache clearing notification to the NFS server where the faulty NFS service occurs through the master monitoring node, so that the NFS server where the faulty NFS service occurs clears the file lock cache corresponding to the faulty NFS service in the local memory; When the service node in the NFS server is a slave monitoring node in the distributed file system, the method further includes: When the cache clearing notification is received, the file lock cache corresponding to the faulty NFS service stored in the local memory is cleared.

5. The method according to claim 4, characterized in that This NFS server is configured with at least one virtual IP address; the file lock cache records corresponding to different virtual IP addresses are in different file lock queues; The clearing of the file lock cache corresponding to the faulty NFS service stored in the local memory includes: Increasing the version number of the file lock queue corresponding to the virtual IP address corresponding to the faulty NFS service stored in the local memory by a set value, so that the state of the file lock cache in the file lock queue existing before receiving the cache clearing notification is set to an expired state; The file lock cache in the expired state is cleared at intervals of a specified time period.

6. The method according to claim 1, characterized in that Determining whether the file lock recovery request is legal based on the file lock information recorded by the file extension attribute corresponding to the failed NFS service includes: If the file lock information of the file extended attribute record corresponding to the faulty NFS service contains target file lock information that matches the file lock recovery request information, then the file lock recovery request is determined to be legal; The target file lock information refers to: the file lock information that matches the host identifier of the file lock requester, the process identifier of the file lock requester, the file lock owner identifier, the file identifier, the file lock type, and the file lock status carried in the file lock recovery request; If the target file lock information matching the file lock recovery request information does not exist in the file lock information of the file extended attribute record corresponding to the faulty NFS service, it is determined that the file lock recovery request is illegal.

7. The method according to claim 1, characterized in that After determining that the file lock recovery request is legitimate, the method further includes: A file lock success recovery message is sent to the client that sent the file lock recovery request, so that the client updates local file lock information based on the file lock success recovery message and continues to use the recovered file lock.

8. A device for processing NFS service failure in a distributed file system, characterized in that: The device is applied to an NFS server in a distributed file system, and includes: A persistence module, configured to persistently store the corresponding file lock information in the extended attributes of the file each time a file lock operation is performed on the file; a sending module configured to, when a service node in the present NFS server is a primary monitoring node in the distributed file system, determine, through the primary monitoring node, an alternative NFS server from the distributed file system if an NFS service provided by the NFS server is detected to have failed by the primary monitoring node, and migrate the virtual IP address corresponding to the failed NFS service to the alternative NFS server, so that the alternative NFS server establishes a network connection with each client corresponding to the failed NFS service based on the migrated virtual IP address, and send, through the primary monitoring node, a service recovery notification to each client based on the network connection, thereby triggering each client to send a file lock recovery request to the virtual IP address; When the service node in the NFS server is a slave monitoring node in the distributed file system, monitor whether the NFS service provided by the NFS server fails and notify the master monitoring node; The determination module is used to determine whether the file lock recovery request sent by the client is legal based on the file lock information recorded in the file extension attribute corresponding to the faulty NFS service when the file lock recovery request is received, and trigger the client to continue using the original file lock on the current NFS server if the file lock recovery request is legal.

9. The device according to claim 8, characterized in that The file lock operation includes: locking operation and unlocking operation; The file lock information includes at least: a data range in the file corresponding to the file lock, a type of the file lock, a status of the file lock, a host identifier of the party requesting the file lock, a process identifier of the party requesting the file lock, an identifier of the file lock owner, and a file identifier of the file on which the file lock is executed; and / or, After determining the alternative NFS server, the sending module is further configured to: send a query notification to the alternative NFS server through the primary monitoring node, so that the alternative NFS server finds file lock information of a file extended attribute record corresponding to the faulty NFS service based on the faulty NFS service; When the service node in the NFS server is a slave monitoring node in the distributed file system, the device further includes: The query module is further configured to, upon receiving the query notification, search for file lock information recorded in the file extended attribute corresponding to the failed NFS service based on the query notification, so that when the NFS server receives a file lock recovery request sent by each client corresponding to the failed NFS service, the NFS server determines whether the file lock recovery request is valid based on the searched file lock information; and / or, After determining the replacement NFS server, the sending module is further configured to: send a cache clearing notification to the NFS server where the faulty NFS service occurs through the primary monitoring node, so that the NFS server where the faulty NFS service occurs clears the file lock cache corresponding to the faulty NFS service in the local memory; When the service node in the NFS server is a slave monitoring node in the distributed file system, the device further includes: A cache cleaning module, configured to clear the file lock cache corresponding to the faulty NFS service stored in the local memory upon receiving the cache clearing notification; and / or, This NFS server is configured with at least one virtual IP address; the file lock cache records corresponding to different virtual IP addresses are in different file lock queues; The clearing of the file lock cache corresponding to the faulty NFS service stored in the local memory includes: Increasing the version number of the file lock queue corresponding to the virtual IP address corresponding to the faulty NFS service stored in the local memory by a set value, so that the state of the file lock cache in the file lock queue existing before receiving the cache clearing notification is set to an expired state; At intervals of a specified time period, the file lock cache in the expired state is cleared; and / or, Determining whether the file lock recovery request is legal based on the file lock information recorded by the file extension attribute corresponding to the failed NFS service includes: If the file lock information of the file extended attribute record corresponding to the faulty NFS service contains target file lock information that matches the file lock recovery request information, then the file lock recovery request is determined to be legal; The target file lock information refers to: the file lock information that matches the host identifier of the file lock requester, the process identifier of the file lock requester, the file lock owner identifier, the file identifier, the file lock type, and the file lock status carried in the file lock recovery request; If the target file lock information matching the file lock recovery request information does not exist in the file lock information of the file extended attribute record corresponding to the faulty NFS service, then the file lock recovery request is determined to be illegal; and / or, After determining that the file lock recovery request is legitimate, the sending module is further configured to: A file lock success recovery message is sent to the client that sent the file lock recovery request, so that the client updates local file lock information based on the file lock success recovery message and continues to use the recovered file lock.

10. An electronic device, characterized in that: The electronic device includes: processor; and A computer-readable storage medium having computer program instructions stored therein, wherein the computer program instructions, when executed by the processor, cause the processor to perform the steps of the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer program instructions, which, when executed by a processor, enable the processor to perform the steps of the method according to any one of claims 1 to 7.

Images