A connection recovery method, device and storage server

Abstract

The application provides a connection recovery method, device and storage server. The method comprises: a client establishing a first connection with a master server of a storage server cluster; the client performing first data transmission with the server cluster through the first connection; in response to receiving a first exception message sent by a backup server in the storage server cluster, the client establishing a second connection with the backup server in the storage server cluster, wherein the first exception message is about the unavailability of the first connection, and the storage server cluster comprises one or more backup servers; and the client resuming the transmission of the first data through the second connection. Thus, the backup server detects the failure of the master server and then sends the failure information to the client. After receiving the failure information, the client directly resumes the connection with the storage server cluster without waiting for a long time.

Description

Technical Field

[0001] This application relates to the field of network communication technology, and in particular to a connection restoration method, apparatus, and storage server. Background Technology

[0002] In a Linux environment, NFS clients can access files stored on an NFS server and share data through the Network File System (NFS) protocol.

[0003] However, when a connection failure occurs between the NFS client and the NFS server, the NFS client cannot immediately confirm the fault after detecting the network anomaly. Instead, it needs to wait for a timeout and retry period before it can truly confirm the occurrence of the fault. The default timeout and retry mechanism involves waiting time, resulting in a relatively high latency for the NFS client when detecting a fault, especially under poor network conditions or slow server response.

[0004] Therefore, there is an urgent need for a solution that allows NFS clients to quickly and promptly detect faults and restore connections. Summary of the Invention

[0005] This application provides a connection recovery method, apparatus, and storage server, which enables the backup server to report fault information to the client when the connection between the client and the primary server is unavailable and affects data transmission. The client can confirm the occurrence of the fault without waiting for the timeout retry period and immediately restore the connection to the storage service cluster.

[0006] In a first aspect, a connection recovery method is provided, applied to a client. The method includes: the client establishing a first connection with the master server of a storage server cluster; the client transmitting first data with the server cluster through the first connection; in response to receiving a first exception message sent by a backup server in the storage server cluster, the client establishing a second connection with the backup server in the storage server cluster, wherein the first exception message relates to the unavailability of the first connection, and the storage server cluster includes one or more backup servers; and the client resuming the transmission of the first data through the second connection.

[0007] The method provided in this application involves a backup server detecting a fault in the primary server and then sending fault information to the client. Upon receiving the fault information, the client directly restores its connection to the server node. In other words, the client node no longer needs to confirm the fault according to the timeout reconnection mechanism, thus eliminating the delay caused by this mechanism and enabling rapid link switching without waiting for a long time.

[0008] In one possible implementation, the method further includes: the backup server establishing the second connection accesses the first data by accessing the storage medium where the first data is stored through the master server; or the backup server establishing the second connection skips the master server and accesses the first data by directly accessing the storage medium where the first data is stored.

[0009] In this implementation, the embodiments of this application can be applied to the deployment of various storage server clusters, such as the primary server and the backup server sharing a remote storage device, or using the primary server's local storage. In different failure scenarios, for example, if the NFS service in the NFS server is normal, the primary server's NFS service provides shared data to the backup server; if the NFS service in the NFS server is unavailable, the backup server directly obtains the shared data.

[0010] In one possible implementation, the method further includes: the backup server stores backup data of the primary server; after the client establishes a second connection with the backup server in the storage server cluster, the transmission of the first data is restored based on the backup data; or after the client establishes a second connection with the backup server in the storage server cluster, the backup server accesses the storage medium where the first data is located and restores the transmission of the first data.

[0011] In this implementation, the backup server backs up the data on the primary server and takes over the NFS service when the connection between the primary server and the client becomes unavailable. Alternatively, the backup server does not back up the data on the primary server but directly accesses the primary server's storage device and takes over the NFS service when the connection between the primary server and the client becomes unavailable.

[0012] In one possible implementation, the method further includes: the backup server detecting an anomaly in the primary server, and when an anomaly is detected in the primary server, the backup server sending a first anomaly message to the client.

[0013] In this implementation, in the event of a primary server failure, the backup server notifies the client in reverse, instructing the client to complete a rapid switchover. This allows the client to confirm the occurrence of the failure without waiting for a timeout retry period.

[0014] In one possible implementation, the establishment of the second connection includes: the backup server that establishes the second connection with the client is bound to a virtual IP, and the virtual IP was originally the virtual IP used by the primary server and the client to communicate. The client and the backup server establish the second connection through the virtual IP.

[0015] In this implementation, the primary server and the backup server restore the connection between the backup server and the primary server by sharing a virtual IP.

[0016] In one possible implementation, the method further includes: configuring multipath access between the client and the storage server cluster, wherein the multipath access includes: paths between the client and each of the multiple storage servers in the storage server cluster that have backup relationships; and establishing a second connection between the client and the backup server in the storage server cluster, including: establishing an available path in the multipath access as the second connection.

[0017] In this implementation, the embodiments of this application can be applied not only to standard NFS clients but also to NFS multipath clients. In an NFS client configured with multipath, when the connection between the client and the server becomes unavailable, the client selects one of the multipath paths to reconnect, restoring the connection between the client and the storage server cluster.

[0018] In one possible implementation, the method further includes: establishing an exception message notification connection between the client and each server in the storage server cluster; and sending a first exception message through the exception message notification connection.

[0019] In this implementation, an exception message notification connection is established between the client and server after the initial connection is established. This connection then uploads server fault information. If the initial connection becomes unavailable, the fault information can be quickly uploaded to the client, allowing the client to immediately restore its connection to the server without waiting for a timeout.

[0020] In one possible implementation, the method further includes: after acquiring the file lock corresponding to the first data, the client performs first data transmission with the storage cluster through the first connection; the client resumes the transmission of the first data through the second connection, including: the client responds to receiving the file lock sent by the backup server; and resumes the transmission of the first data through the second connection.

[0021] In this implementation, when resuming the first data transmission, the backup server can carry the file lock of the first data. Therefore, the client does not need to re-acquire the file lock, avoiding the additional network overhead and latency caused by re-acquiring the lock. It also supports more customized message interactions in the future, improving overall system performance.

[0022] In one possible implementation, the method further includes: the client subscribing to abnormal events in the server cluster; the abnormal event types include master server failure, master server network failure, link failure, and link switching during operation and maintenance.

[0023] In this implementation, the client subscribes to the server's failure events. After a failure event occurs, the server can obtain the failure information and send it to the client through the subscription mechanism, so that the client node can quickly confirm the first connection failure and then restore the connection to the storage server cluster.

[0024] Secondly, a link switching method is provided, applied to a storage service cluster. The method includes: in response to a first connection establishment request sent by a client, the master server in the storage service cluster establishes a first connection with the client; providing the client with the transmission of first data through the first connection; after detecting an anomaly in the first connection, sending a first anomaly message to the client through another connection, the first anomaly message indicating that the first connection is unavailable; in response to a second connection establishment request sent by the client, the backup server of the master server establishes a second connection with the client, and resumes the transmission of the first data through the second connection.

[0025] Thirdly, embodiments of this application provide a connection recovery device applied to a client. The device includes: an establishment module, used for the client to establish a first connection with the master server of a storage server cluster; and further used to establish a second connection with a backup server in the storage server cluster in response to receiving a first exception message sent by a backup server in the storage server cluster, wherein the first exception message relates to the unavailability of the first connection, and the storage server cluster includes one or more backup servers; a transmission module, used for the client to perform a first data transmission with the server cluster through the first connection; and a recovery module, used for the client to recover the transmission of the first data through the second connection.

[0026] Fourthly, embodiments of this application provide a connection recovery device applied to a storage service cluster. The device includes: an establishment module, configured to establish a first connection between the master server in the storage service cluster and the client in response to a first connection establishment request sent by a client; further configured to send a first exception message to the client through another connection after detecting an anomaly in the first connection, the first exception message indicating that the first connection is unavailable; a transmission module, configured to provide the client with the transmission of first data through the first connection; and a recovery module, configured to establish a second connection between the master server and the client through a backup server in response to a second connection establishment request sent by the client, and restore the transmission of the first data through the second connection.

[0027] Fifthly, a storage server is provided, including a memory and a processor, wherein the memory stores instructions that, when executed by the processor, cause the method provided in the first aspect or the method provided in the second aspect to be implemented.

[0028] In a sixth aspect, embodiments of this application provide a computer storage medium storing instructions that, when executed on a computer, cause the computer to perform the method provided in the first aspect, or to perform the method provided in the second aspect.

[0029] In a seventh aspect, embodiments of this application provide a computer program product containing instructions that, when executed on a computer, cause the computer to perform the method provided in the first aspect, or to perform the method provided in the second aspect.

[0030] The beneficial effects of aspects two through seven can be found in the above description of the beneficial effects of aspect one, and will not be repeated here. Attached Figure Description

[0031] Figure 1 This is a schematic diagram illustrating fault recovery in the standard NFS client protocol of related technologies.

[0032] Figure 2 A schematic diagram of a network storage system architecture for an NFS standard client is provided in an embodiment of this application;

[0033] Figure 3 This is a schematic diagram illustrating a connection restoration method provided in an embodiment of this application.

[0034] Figure 4 A schematic diagram illustrating another connection restoration method provided in this application embodiment;

[0035] Figure 5 A schematic diagram of the structure of a network storage system based on an NFS multipath client is provided in an embodiment of this application;

[0036] Figure 6 A schematic flowchart illustrating a connection restoration method provided in an embodiment of this application;

[0037] Figure 7 This application provides a schematic diagram of a first connection and an exception message notification connection establishment process.

[0038] Figure 8 This is a schematic diagram of a second connection establishment process provided in an embodiment of this application;

[0039] Figure 9 A schematic flowchart illustrating a connection restoration method provided in an embodiment of this application;

[0040] Figure 10 This application provides a schematic diagram of a multi-site active-active system.

[0041] Figure 11 This is a schematic diagram of the connection restoration device provided in the embodiments of this application;

[0042] Figure 12 This is a schematic diagram of the connection restoration and transmission device provided in the embodiments of this application;

[0043] Figure 13 A storage server provided in an embodiment of this application;

[0044] Figure 14 A storage server cluster is provided as an embodiment of this application;

[0045] Figure 15 This is a schematic diagram of another storage server cluster structure provided in an embodiment of this application. Detailed Implementation

[0046] The solutions provided in the embodiments of this application will now be described with reference to the accompanying drawings. In the embodiments of this application, "multiple" refers to two or more objects, and "various types" refers to two or more types. Terms such as "first," "second," etc., are only used to distinguish similar objects and are not necessarily used to describe a specific order or number of objects.

[0047] To facilitate understanding of the solutions provided in the embodiments of this application, the technical terms that may be involved in the embodiments of this application will be introduced first.

[0048] Linux operating system: An open-source operating system based on the Unix design philosophy, consisting of a kernel and user space. The kernel is the core of the system, responsible for basic functions such as managing hardware resources, process scheduling, memory management, and the file system. User space contains all applications running on the operating system; it interacts with the kernel through system calls and utilizes services provided by the kernel, such as file operations.

[0049] Unix systems: Operating systems based on the Unix kernel. Widely used in servers, workstations, and embedded devices.

[0050] Virtual File System (VFS): Located in the Linux kernel, it provides a unified abstraction layer for different types of file systems. It determines which type of driver (e.g., NFS driver, ext4 driver, etc.) should be called at the file system layer.

[0051] Storage Area Network (SAN): A high-speed network specifically designed to connect storage devices and servers, providing high-performance, high-availability, and scalable storage solutions.

[0052] Ceph Storage Cluster (Ceph): A high-performance, highly scalable, and highly available distributed storage system that supports multiple storage interfaces such as object storage, block storage, and file storage.

[0053] Serial Attached SCSI (SAS): A high-speed serial interface technology used to connect storage devices (such as hard drives and solid-state drives) to computers, providing high performance, high reliability and flexible scalability.

[0054] Remote Procedure Call (RPC): A technique that allows a program to call a procedure on a remote server over a network, just as if it were a local procedure, thereby enabling communication between systems.

[0055] Network File System (NFS): Allows clients to access source files on a server over a network in a manner similar to accessing local files. It operates on a client-server architecture, where the client requests file system resources from the server over the network. In Linux systems, the NFS client is the terminal device using the NFS service; it achieves transparent access to remote data by mounting the file system on the remote server. The NFS kernel module is the core of the NFS client, responsible for parsing and processing NFS protocol requests.

[0056] Transmission Control Protocol (TCP) is a connection-oriented, reliable, byte-stream-based transport layer communication protocol. TCP establishes a connection through a three-way handshake and employs mechanisms such as acknowledgment, retransmission, ordering, and sliding windows to ensure the reliability of data transmission. It is widely used for internet data transmission, ensuring that data can be transmitted accurately and completely between different network devices.

[0057] File locks: In an NFS master-slave system, file locks are used to control concurrent access to shared files by multiple NFS clients to avoid data conflicts and inconsistencies. NFSv4 natively supports file locks, ensuring the reliability and consistency of lock operations by maintaining file lock states. During master-slave failover, file lock states need to be correctly synchronized to avoid access conflicts caused by inconsistent lock states.

[0058] Virtual IP Address (VIP): An IP address that is not directly bound to a specific physical device or network interface. It is typically used to implement functions such as high availability, load balancing, and failover.

[0059] NFS Multipath Client: An optimization technique for the NFS protocol used to establish multiple network paths between the NFS client and server to achieve load balancing and failover. The client needs to be configured with NFS client software that supports multipathing. The client distributes I / O requests across multiple network paths using multipathing techniques (such as round-robin algorithms).

[0060] The Mount command is used to mount a file system, attaching a specified storage device or network share (such as an NFS share) to a specified mount point, making it available in the current file system.

[0061] Network protocol stack: A set of protocol layers in a computer network, responsible for data encapsulation, transmission, and decapsulation. It is typically divided into a five-layer model (TCP / IP) or a seven-layer model, with each layer performing a specific function, ranging from the physical transmission at the data link layer to the specific service interaction at the application layer. Each layer collaborates with the layers above and below to ensure that data can be transmitted effectively and securely from the source to the destination.

[0062] Transmission Control Protocol / Internet Protocol (TCP / IP) is a set of communication protocols used to connect devices to the Internet or other networks. TCP / IP provides rules and standards for data transmission and reception.

[0063] Internet Protocol Address (IP Address): A unique numerical identifier assigned to devices on a network. Solid State Drive (SSD): A storage device based on semiconductor storage technology that uses flash memory chips (such as NAND Flash) to store data.

[0064] Multi-site active-active: This is a high-availability architecture that refers to a system deployed in multiple geographical locations (sites) in which all sites are active and provide services to the outside world at the same time.

[0065] Disaster recovery environment: A solution that ensures business continuity by deploying redundant systems in different locations to achieve failover and data backup.

[0066] VDBENCH: An open-source I / O workload generator developed by Oracle, used to verify the performance and data integrity of storage systems. It supports various parameter configurations and can simulate different read and write loads, making it suitable for testing the performance of file systems and block devices.

[0067] Input / Output (I / O): The input and output operations of data in a computer system.

[0068] Socket connection: is a communication mechanism in computer networks that allows programs on different computers to transmit data over the network. Through a socket, a program can establish a connection, send and receive data, thereby enabling bidirectional communication between the client and the server.

[0069] Synchronize Sequence Numbers (SYN) packets: These are the synchronization sequence numbers used in the TCP protocol as handshake signals to establish a connection.

[0070] The Synchronize-Acknowledge (SYN-ACK) packet is a signal packet used in the second phase of the TCP protocol to establish a connection.

[0071] Acknowledgment (ACK) packet: This is a signal packet in the TCP protocol used to acknowledge receipt of data.

[0072] In Linux environments, file communication primarily relies on the Network File System (NFS) protocol. The NFS protocol enables remote file access between NFS clients and NFS servers through a network protocol stack, allowing Linux systems to mount remote file systems over the network. Through the NFS protocol, NFS servers can provide storage resources to NFS clients in a network-shared manner, while NFS clients perform file read / write, mounting, and other operations using the NFS protocol. The NFS protocol facilitates data transfer and access between NFS clients and NFS servers, making it a common network file sharing solution in Linux environments.

[0073] In an NFS system, an NFS client sends file operation requests to the NFS server, which processes the requests and returns the results. When a connection failure occurs between the NFS client and the NFS server, related technologies rely on NFS driver timeout detection to determine the connection failure. Figure 1 This is a schematic diagram illustrating fault recovery in the standard NFS client protocol of related technologies. For example... Figure 1As shown, if the network connection between the NFS client and the NFS master server is interrupted (e.g., the network link between the client and server is broken or packet loss occurs), the NFS client will still attempt to establish a connection with the NFS master server even if it fails to detect the network failure. However, because the network connection is interrupted, the connection between the client and the master server is unavailable, causing the client's request to fail to be sent to the master server. The NFS client waits for a response from the master server within a timeout period; only if no response is received after the timeout will the connection be considered faulty. The specific value of the timeout period depends on the NFS client's kernel configuration and mount options; the timeout period varies depending on the operating system and driver software, but it is usually around 15 seconds. In other words, only after the timeout retry period can the NFS client truly confirm the occurrence of the fault and attempt to restore the connection.

[0074] For example, when an NFS client node sends a request to the NFS master server, if it does not receive a response within the set timeout period (e.g., 3 seconds), the NFS client will initiate a retry mechanism. Each retry involves the client first waiting 3 seconds for a response. If there is still no response, it will wait a 2-second retry interval before resending the request. This process takes a total of 5 seconds (3-second timeout + 2-second retry wait). Depending on the NFS client configuration, the NFS client will also wait a total of 15 seconds (5 seconds + 5 seconds + 5 seconds) after 3 retries before determining that the connection between the NFS client and the NFS master server has failed, truly confirming the failure, and then restoring the failed connection.

[0075] Furthermore, if multiple reconnection attempts fail, the NFS client will gradually increase the retry waiting time based on a penalty mechanism (such as exponential backoff). Therefore, how to enable the NFS client to detect connection failures in a timely and accurate manner, triggering the NFS client to restore the connection with the storage server cluster as early as possible, eliminating the need for fault confirmation according to the timeout reconnection mechanism, and reducing the latency caused by the timeout reconnection mechanism, is a technical problem that urgently needs to be solved.

[0076] This application provides a connection recovery method, including: a client establishing a first connection with the master server of a storage server cluster; the client transmitting first data with the server cluster through the first connection; in response to receiving a first exception message sent by a backup server in the storage server cluster, the client establishing a second connection with the backup server in the storage server cluster, wherein the first exception message relates to the unavailability of the first connection, and the storage server cluster includes one or more backup servers; the client resuming the transmission of the first data through the second connection. Thus, NFS client nodes can promptly and accurately detect connection failures with the master server of the storage server cluster and immediately restore the connection with the storage server cluster. Storage servers manage local or remote persistent storage media (e.g., SSDs or HDDs) to achieve persistent data storage. Similar to general-purpose servers, storage servers also have processors and memory.

[0077] Figure 2 This is a schematic diagram of the structure of a network storage system based on an NFS standard client, provided as an embodiment of this application. Figure 2 As shown, the network storage system 200 includes an NFS client and a storage server cluster. The storage server cluster comprises multiple storage nodes connected via a network to form a storage resource pool for centralized data storage and management. This pool can be constructed using distributed storage technologies (such as Ceph) or traditional storage arrays (such as Storage Area Networks (SANs)). Storage nodes can be physical servers, virtual machines, or storage devices.

[0078] An NFS server is a service node in a storage server cluster that provides storage resources to clients via the NFS protocol. These storage resources can be shared storage provided by the storage server cluster or local storage on the NFS server itself. For example, an NFS server can directly use its locally connected storage devices (such as hard drives or solid-state drives, SSDs) as storage resources and share these local storage directories with clients via the NFS protocol. Alternatively, the NFS server can mount shared storage resources provided by the storage service cluster (such as the distributed file system Ceph or the centralized shared file system SAN).

[0079] In a network of NFS servers, the one currently transmitting data with the NFS client is the primary NFS server. For example, the primary server is bound to a virtual IP address, while the backup server is in monitoring mode. When the primary server fails, the backup server takes over the virtual IP address and continues to provide service.

[0080] In one possible implementation, the NFS primary and secondary servers can share storage resources. They share storage via a network connection and access the same data. Thus, in the event of a primary server failure, the secondary server takes over the NFS service by directly accessing the shared storage resources, providing remote file services to clients. For example, the storage server cluster uses a SAN to implement storage services. The NFS primary and secondary servers access shared storage resources. The NFS server mounts the storage resources provided by the SAN, making them part of its local file system and providing them to clients via the NFS protocol.

[0081] In another possible implementation, the storage resources provided by the NFS primary and backup servers can also be their own local storage resources (such as hard drives, SSDs), etc. As mentioned above, the client mounts a shared directory on the primary server, where all or part of the data is the first data on the primary server that the client wants to access. The backup server, for example, backs up the data on the primary server through a synchronization mechanism. In this way, when the primary service fails, the backup server can take over the NFS service by accessing the backup data, providing remote file services to the client.

[0082] For example, when the NFS server provides its own local storage resources, such as a primary server and a backup server sharing a storage device via Serial Attached SCSI (SAS), the primary server is configured with SAS storage devices, and each backup server is connected to the primary server's SAS storage device. The backup servers can directly access the data on the SAS storage devices, thus enabling the primary and backup servers to share storage resources.

[0083] NFS clients are deployed on compute nodes that need to access shared storage resources. They access the shared file system of the NFS server over the network. Specifically, by mounting shared directories on the NFS server, the remote file system is mapped to the local machine, allowing files in the storage server cluster to be accessed as if they were local files. Compute nodes can be physical servers, virtual machines, or containers, and the server's operating system can be Linux. Figure 2 The examples shown are merely illustrative; there can be any number of clients and servers.

[0084] Clients access remote shared file systems provided by NFS master servers via the NFS protocol. Specifically, when a user-space application (such as a command-line tool) initiates an access request to an NFS shared file, the operating system kernel first triggers a mount operation on the NFS file system through a system call interface. After successful mounting, subsequent file read and write operations are performed.

[0085] System calls pass file mount requests (such as the `mount` command) to the kernel's Virtual File System (VFS). VFS provides a unified interface for different file system types (such as local file systems and network file systems). If it's a remote file system mounted via NFS, VFS forwards the request to the NFS client kernel module. The NFS client is a module in the operating system kernel that handles remote file system access. The NFS client uses the interface provided by the NFS client kernel module to generate requests and interact with the NFS master server. The NFS client kernel module handles communication between the NFS client and the NFS master server, constructing the request based on mount information (such as the NFS master server's Internet Protocol Address (IP address), shared directories, etc.). The NFS client kernel module then passes the request to the NFS driver, which encapsulates the request as a Remote Procedure Call (RPC) request and sends it to the NFS master server via a network protocol stack such as Transmission Control Protocol / Internet Protocol (TCP / IP). The NFS client sends a mount request through the first connection, and the first connection establishment module establishes the first connection. For example, if the first connection is a TCP connection, during the TCP connection establishment process, the NFS client completes the connection with the NFS master server through a three-way handshake. Specifically, the client sends a Synchronize Sequence Numbers (SYN) packet to the master server, the master server responds with a Synchronize-Acknowledge (SYN-ACK) packet, and the client then returns an Acknowledgement (ACK) packet, thus completing the TCP connection establishment.

[0086] After the initial connection is established, the NFS client sends a mount request to the NFS master server via RPC and completes the mount operation upon receiving a response. Following this, the NFS client and NFS master server perform the first data transfer over the initial connection. This first data includes the client's operation requests and responses to shared files on the master server; for example, when the client reads a file, the NFS master server transfers the file content to the client; when the client writes to a file, it sends the data to the master server for storage.

[0087] In this embodiment of the application, after the first connection is established, an exception message notification connection is also established between the NFS client and the NFS server through an exception message notification connection establishment module. This exception message notification connection includes at least one connection. The NFS client establishes an exception message notification connection with each NFS server respectively. For example, the exception message notification connection is a TCP connection. The establishment of this TCP connection will be described in detail later.

[0088] After the exception message notification connection is established, the NFS client sends subscription / registration messages to each NFS server through the first exception message subscription module. The subscription messages include exception events from the NFS master server.

[0089] The anomaly collection module handles the collection of anomaly information. For example, it collects anomaly information about the NFS master server from the heartbeat detection module. For instance, if the master server does not respond to a heartbeat signal (e.g., the master server is down), the standby server will assume the master server is faulty. For example, the standby server detects network faults by monitoring the master server's network interface card (NIC). The standby server can periodically ping the master server's NIC IP address or check the availability of service ports to confirm the master server's network status and thus determine if the first connection to the master server is available.

[0090] After subscribing to NFS master server exception messages, the exception events of the NFS master server are collected by the exception collection module of the NFS backup server and sent to the NFS client through the first exception message sending module. This process can usually be completed within 2 seconds.

[0091] After receiving the first exception message, the first exception message receiving module of the NFS client notifies the second connection establishment module to establish a second connection. This second connection is between the NFS client and the NFS standby server. The NFS standby server is determined before the second connection is established. For example, the NFS primary server and the NFS standby server monitor each other's status. The NFS primary server and the NFS standby server monitor each other's status through a heartbeat detection module. The heartbeat detection module on the standby server periodically sends heartbeat signals to the primary server. If the standby server detects a loss of the primary server's heartbeat signal, it considers the primary server to have failed and triggers a failover mechanism. The virtual IP is migrated to the standby server, and the standby server takes over the NFS service from the primary server. For example, the primary server configures a virtual IP and binds it to its own network interface. Clients access the NFS primary server through the virtual IP address. When the primary server malfunctions and the standby server does not receive a heartbeat signal from the primary server within a preset time, the standby server considers the primary server unavailable. The backup server binds to the virtual IP address and, for example, sends an ARP broadcast packet to the network to notify devices in the network that the MAC address corresponding to the virtual IP address has been updated to the backup server's MAC address. When a client accesses the virtual IP address again, the data packet is sent to the backup server bound to that virtual IP.

[0092] It's important to note that, as mentioned earlier, virtual IPs can be used to share IP addresses between primary and backup servers. This allows clients to communicate with the server using the same virtual IP address, without needing to know which server is primary or backup. Both the primary and backup servers can be configured with the same virtual IP address. When the primary server fails, the backup server takes over the VIP address, and the client always communicates with the same VIP address. Alternatively, the primary and backup servers can be configured with the same VIP address and listen on the same service port. The client communicates with the NFS service using the virtual IP address and port number. From the client's perspective, it appears to be connected to the same NFS server. Therefore, the newly established second connection can be seen as a recovery of an invalid first connection. In reality, the actual IP address of the NFS server the client is connecting to changes, and the physical link and IP routing between the NFS client and NFS server may also change; therefore, this can also be viewed as a link switch between the NFS client and NFS server.

[0093] Figure 3 This is a schematic diagram illustrating a connection restoration method provided in an embodiment of this application. Figure 3As shown, when an NFS client node first requests to mount a remote file system, the client establishes an initial connection with the master server. After the initial connection is established, the NFS client will establish at least one additional exception message notification connection with the NFS master server. These exception message notification connections include the newly established exception message notification connection between the NFS client and the NFS master server, as well as the newly established exception message notification connection between the client and the NFS backup server.

[0094] For example, an NFS client uses the `mount` command to mount the NFS shared directory ` / mnt / nfs`, requesting to mount the ` / shared` directory on a remote NFS master server. For instance, the NFS master server's IP address is 192.168.1.100, and it listens on port 1000. During the initial connection establishment process, the NFS client creates a connection, for example, via a socket, specifying the target NFS master server's IP address and port number. For example, the NFS client's IP address might be 192.168.1.50, and the port number might be 2000. The NFS client then establishes the initial connection with the NFS master server through a TCP three-way handshake.

[0095] This application embodiment establishes a new exception message notification connection by obtaining the IP address of the target NFS server for establishing the exception message notification connection. This IP address is, for example, returned by the NFS master server based on user configuration. On the established first connection, the NFS client can read the NFS master server's configuration file, which includes the NFS master server's IP address and the IP addresses of the NFS backup servers in a master-slave relationship.

[0096] For example, in an NFS primary server and an NFS backup server, the IP address of the other is typically specified in the configuration file. The configuration file on the NFS primary server lists the IP address of the NFS backup server. The NFS primary server can access the NFS backup server through its IP address. This application embodiment does not impose any limitations on this.

[0097] After obtaining the IPs of the NFS primary and NFS backup servers, the NFS client establishes exception message notification connections with the NFS primary and NFS backup servers respectively. For example, the exception message notification connection is a TCP connection, and each TCP connection is identified by the source client IP address, source client port number, target server IP address, and target server port number.

[0098] For example, after an error message notifies the connection to be established, the NFS client and NFS server maintain a stable TCP connection through a heartbeat mechanism. The NFS client sends a heartbeat message to the NFS server at preset intervals, and the NFS server responds upon receiving it. If the NFS client or NFS server does not receive a heartbeat message or response within the expected time, it assumes the connection has been broken, and the client will send an error message to notify the connection to reconnect.

[0099] In this embodiment, after the virtual IP address of the NFS primary server is switched to the NFS backup server, an exception event is sent to the NFS client through the previously established exception message notification connection. Upon receiving the exception event, the NFS client immediately establishes a second connection with the NFS backup server. After the second connection is established, the NFS backup server becomes the NFS primary server.

[0100] In one possible implementation, in some versions of the NFS protocol, after establishing a second connection, the NFS client sends a request to the NFS standby server using a saved file handle, requesting the resumption of previous file operations. Upon receiving the client's request, the NFS standby node verifies and retrieves the file's status information using the file handle. Since the file handle points to a specific file or directory on the server, the NFS client can then continue reading, writing, or performing other file operations, thus restoring the previous session state.

[0101] It should be noted that a storage server cluster can also include one NFS master server and multiple NFS backup servers. Figure 4 This is a schematic diagram illustrating another connection restoration method provided in an embodiment of this application. Figure 4 As shown, the NFS client accesses the services provided by the NFS master server over the network. If the NFS master server fails, one of the multiple NFS backup servers will take over the NFS master server to provide services to the client; this embodiment does not impose any restrictions on this.

[0102] For example, in a system with multiple standby servers, each NFS standby server is assigned a priority. A standby server with higher priority will take over the virtual IP address first if the primary server fails. That is, when an NFS client initiates a request using the virtual IP address, the higher-priority standby server will respond. The NFS client can continue to access the higher-priority standby server using that virtual IP address.

[0103] After the NFS client and the NFS master server establish the initial connection, the NFS client will establish exception message notification connections with all NFS servers. If the NFS master server fails, making the initial connection between the NFS client and the NFS master server unavailable, the IP address of the NFS master server will switch to one of the NFS backup servers, for example, the backup server with the highest priority. The NFS master server will send a first exception message through the established exception message notification connection with the NFS client, notifying the NFS client that the initial connection is unavailable. Upon receiving the first exception message, the NFS client will immediately establish a second connection with the NFS backup server and resume the transmission of the initial data.

[0104] like Figure 4 As shown, the NFS master server and each NFS standby server are connected via a network. The NFS standby server can access the storage resources of the NFS master server, and the storage resources can be local resources and remote shared storage resources.

[0105] The above describes the process of restoring the first connection in a standard NFS primary / standby system according to an embodiment of this application. In another scenario, besides a standard NFS primary / standby system, embodiments of this application can also be applied to a network storage system based on an NFS multipath client. Figure 5 This is a schematic diagram of a network storage system based on an NFS multipath client, provided as an embodiment of this application. (See diagram below.) Figure 5 As shown, the NFS multipath client is configured to communicate with the NFS master server via multiple network paths. The NFS client accesses the NFS master server through these multiple paths. For example, path 1 and path 2 are two different network paths between the client and the master server. The NFS client can use path 1 and path 2 to establish a connection with the master server. The NFS client can also be configured to communicate with the NFS backup server via multiple paths.

[0106] When an NFS client first requests to mount a remote file system, the NFS client establishes an initial connection with the NFS master server via path 1 or path 2. After the initial connection is established, the NFS client node will establish additional exception message notification connections with both the NFS master and NFS standby nodes. These exception message notification connections include those with the NFS master and standby servers. If the NFS master server fails, either the master or standby server will send a first exception message via the exception message notification connection. This first exception message indicates that the initial connection between the NFS client and the NFS master server is unavailable. The NFS client immediately acknowledges the failure of the first connection and begins the reconnection process.

[0107] The NFS client can restore the first connection by choosing a second path between itself and the NFS primary server, or a single path between itself and the NFS backup server.

[0108] In one possible implementation, if a multipath connection is also established between the NFS server backup node and the NFS client, the NFS client will choose to reconnect to one of the following: path 2 of the NFS primary server, path 1 of the NFS backup server, or path 2 of the NFS backup server. In other words, one available path in the multipath access is established as the second connection. This application does not impose any restrictions on this.

[0109] For example, assume the client has two network interfaces: eth0 and eth1. The IP address on eth0 is 192.168.1.100, and the IP address on eth1 is 192.168.1.102. The server also has two network interfaces: eth0 and eth1. The IP address on eth0 is 192.168.1.200, and the IP address on eth1 is 192.168.1.201. The client establishes a path 1 connection by connecting to the server's eth0 (IP address 192.168.1.200) via eth0 (IP address 192.168.1.100), and a path 2 connection by connecting to the server's eth1 (IP address 192.168.1.201) via eth1 (IP address 192.168.1.102). Upon initial mount, the NFS client establishes the first connection via path 1 for initial data transfer.

[0110] Subsequently, the NFS client establishes exception message notification connections with both the NFS master server and the NFS standby server. When the NFS client accesses the NFS master server via path 1, for example, if path 1 fails and the first TCP connection on path 1 becomes unavailable, the NFS standby server will detect this failure and send the first exception message to the NFS client via the normal message notification connection.

[0111] After receiving the first exception message, the NFS client parses it and confirms that the first connection is unavailable. The NFS client then establishes a second connection. This second connection can be established via path 2 between the NFS client and the NFS master server, or between the NFS client and the NFS backup server.

[0112] based on Figure 2 and Figure 5 The system shown illustrates the connection restoration method provided in the embodiments of this application. Figure 6 This is a flowchart illustrating a connection restoration method provided in an embodiment of this application. Figure 6As shown, the connection restoration method provided in this application embodiment includes the following steps S601 to S604.

[0113] S601, the client establishes the first connection with the master server of the storage server cluster.

[0114] In one possible implementation, after the initial connection is established, an exception message notification connection is established between the client and each of at least one of the servers.

[0115] Figure 7 This is a schematic diagram illustrating a first connection and an exception message notification connection establishment process provided in an embodiment of this application. Figure 7 As shown, the NFS client establishes the first connection with the NFS master server. The NFS client initiates the first connection request through the network protocol stack. For example, the client uses source port number A2 to initiate the first connection request. The NFS master server listens on target port number B1, waiting for the NFS client to connect. The NFS client sends a SYN (synchronization) packet to the NFS server, requesting to establish the first connection. After receiving the SYN packet, the server sends a SYN-ACK packet to the NFS client, indicating its agreement to establish the connection. After receiving the SYN-ACK packet, the NFS client confirms that the NFS master server agrees to establish the connection and sends an ACK (acknowledgment) packet to the NFS master server, indicating that the first connection has been successfully established.

[0116] After the initial connection is established, the NFS client initiates an exception message notification connection request via, for example, the TCP / IP protocol stack. For instance, the client uses source port A1 to initiate the request. The NFS master server listens on target port B2, waiting for the NFS client to connect. The NFS client sends a SYN (synchronization) packet to the NFS master server, requesting the establishment of an exception message notification connection. Upon receiving the SYN packet, the server sends a SYN-ACK packet to the NFS client, indicating its agreement to establish the connection. Upon receiving the SYN-ACK packet, the NFS client confirms the NFS master server's agreement and sends an ACK (acknowledgment) packet to the NFS master server, indicating that the exception message notification connection has been successfully established. Similarly, exception message notification connections are established between the NFS client and the NFS backup server.

[0117] In one possible implementation, the types of abnormal events include master server failure, master server network failure, link failure, and link switching during proactive maintenance.

[0118] For example, the availability of a specific NFS server port (e.g., the port for the NFS service) can be used to determine if the NFS master server is functioning correctly. For instance, if the firewall on the master server is blocking connection requests to that port, the NFS service is unavailable.

[0119] For example, periodically check the network reachability of the primary server. For instance, by pinging the primary server's IP address, if the NFS backup server cannot ping the primary NFS server's IP address, it will report a network failure to the NFS client.

[0120] S602, the client transmits data to the server cluster through the first connection.

[0121] In one possible implementation, the first data includes client requests and responses to operations on shared files on the master server.

[0122] S603, in response to receiving a first exception message sent by a backup server in the storage server cluster, the client establishes a second connection with the backup server in the storage server cluster, wherein the first exception message relates to the first connection being unavailable, and the storage server cluster includes one or more backup servers.

[0123] In one possible implementation, when an abnormal event occurs, such as an NFS primary server failure, network failure, link failure, or active link switching during maintenance, the NFS backup server will send a first abnormal message to the connection. This first abnormal message indicates that the NFS primary server has experienced the aforementioned failure, rendering the first connection unavailable. Abnormal events include those caused by the primary server itself, preventing it from properly handling NFS client requests. For example, a network card failure on the primary server may prevent the establishment of a network connection. A problem with the primary server's storage media may render data inaccessible. The NFS service on the primary server may be unusable, preventing responses to client requests. The primary server's firewall may block the ports used for communication between the primary server and clients, making client requests unreachable.

[0124] In situations that could cause the first connection to become unavailable, if the NFS service on the NFS server is functioning normally, the NFS server can access its local storage, but it cannot share the storage resources with NFS clients via the NFS protocol. The primary server can then provide its local storage resources to the secondary server. In this way, the secondary server can access the first data through the primary server by accessing the storage medium containing the first data.

[0125] In one possible implementation, if the NFS service fails to provide normal service, the backup server directly accesses the primary server's local storage medium to access the first data.

[0126] In one possible implementation, the NFS client subscribes to NFS server exception events via an exception message notification connection. The NFS client sends a subscription request to the NFS server through this connection, with the exception events of the NFS server as the subscriber. The NFS client is the subscriber, and the NFS server is the publisher. The NFS server processes the subscription request and pushes the first exception message to the NFS client when a subscription event occurs. The NFS server maintains a subscriber list, storing the subscriber's socket and subscribed topic. When an exception event occurs, the NFS server iterates through the subscriber list and pushes the first exception message to each subscriber. For example, if the primary NFS server experiences an exception, the backup NFS server can send the primary NFS server's exception message to the NFS client through the established exception message notification connection.

[0127] As mentioned above, based on the anomaly detection mechanism between the NFS primary and NFS backup servers, if the NFS primary server fails to send a heartbeat signal to the NFS backup server, the NFS backup server will detect the failure of the NFS primary server and take over the virtual IP address. In other words, the virtual IP address will be automatically transferred to the NFS server backup node, allowing NFS clients to continue accessing the NFS backup server through the virtual IP address.

[0128] In one possible implementation, multipath access is configured between the client and at least one server. Multipath access includes: paths between the client and multiple storage servers that have backup relationships in the storage server cluster; and establishing one of the available paths in the multipath access as a second connection.

[0129] S604, the client resumes the transmission of the first data through the second connection.

[0130] In one possible implementation, after acquiring the file lock corresponding to the first data, the client performs the first data transmission with the storage cluster through the first connection; the client resumes the transmission of the first data through the second connection, including: the client responding to receiving the file lock sent by the backup server; and resuming the transmission of the first data through the second connection.

[0131] Figure 8 This is a schematic diagram of a second connection establishment process provided in an embodiment of this application. Figure 8As shown, during the read / write operation between the NFS client and the NFS master server, the NFS client sends a SYN packet through the first connection. If, during the first data transmission, the NFS backup server detects an anomaly in the NFS master server, for example through a heartbeat mechanism, making the first connection unavailable, the NFS backup server sends a first exception message to the NFS client. Upon receiving the first exception message, the NFS client immediately confirms that the first connection is unavailable and does not need to confirm the fault according to the timeout reconnection mechanism. Instead, it directly sends a second connection establishment request to the NFS backup server and establishes the second connection through a three-way handshake.

[0132] In the embodiments provided in this application, when a primary server failure causes the connection between the client and the primary server to become unavailable, the backup server first detects the primary server's failure and then sends failure information to the client. The client does not need to experience timeout retry delays and can directly restore the connection to the storage server cluster. This allows for rapid link switching without waiting for a long period.

[0133] The following test data verifies that the connection recovery time of the NFS client is significantly shortened after adopting the connection recovery method provided in this application embodiment. For example, an input / output (I / O) tool (such as VDBENCH) continuously reads and writes to the file share, and the system's response and the NFS client's ability to recover the connection are tested under a simulated NFS master server failure. The steps include: starting VDBENCH to perform continuous read and write operations; after the NFS master server stops the NFS service, monitoring the NFS client's response to see how long it takes for the NFS client to restore the connection with the NFS backup server. After multiple experiments, the following results were obtained:

[0134] Using the connection recovery method provided in this application embodiment, the time from the first connection failure to connection recovery for the NFS client is less than 2 seconds. Without the connection recovery method provided in this application embodiment, the time from the first connection failure to connection recovery for the NFS client is approximately 15 seconds.

[0135] In some versions of the NFS protocol, file lock status is primarily maintained by the server, with clients requesting and reclaiming locks through communication with the server. When a client needs to perform a write operation on a file (or some read operation requiring exclusive access), it sends a lock request to the NFS master server via the first connection. Upon receiving the lock request, the NFS server checks the current lock status to determine whether to approve the request. If the file is not currently locked by other NFS clients, or if the requested lock type is compatible with existing locks (e.g., multiple shared locks can coexist), the NFS master server approves the lock request and records the lock status on the NFS master server. If the first connection becomes unavailable, the backup server can notify the connection to upload the file lock via an exception message. In this way, the client can avoid the additional network overhead and latency caused by re-requesting the lock.

[0136] The NFS server helps identify and locate network, storage, and other faults by collecting and analyzing diagnostic data from NFS clients. For example, NFS clients log client requests and responses, NFS server response times for processing client requests, error codes (such as insufficient permissions), and information on abnormal or failed requests and service interruptions (such as mount failures, access permission issues, and storage unreachability). Through an error message notification connection established between the NFS client and the NFS server, the NFS server can collect diagnostic data such as NFS client logs to promptly detect network faults, storage faults, file system errors, or client anomalies, diagnose and resolve potential problems, and thus ensure the stable operation of the entire system.

[0137] This application embodiment transmits different messages over the abnormal message notification connection, supporting more customized message interactions in the future and improving the overall system performance.

[0138] Figure 9 This is a flowchart illustrating a connection restoration method provided in an embodiment of this application. Figure 9 As shown, the connection recovery method provided in this application embodiment is applied to a storage server cluster, including the following steps S901 to S904.

[0139] S901, in the storage service cluster, the master server responds to the first connection establishment request sent by the client and establishes the first connection with the client.

[0140] In one possible implementation, after establishing the first connection with the client, the method further includes: in response to an exception message notification establishment request sent by the client, establishing an exception message notification connection with the client; and sending the first exception message through the exception message notification connection.

[0141] S902, the master server provides the client with the first data transmission through the first connection.

[0142] In S903, when a backup server in the storage service cluster that has a backup relationship with the primary server detects an anomaly in the first connection, it sends a first anomaly message to the client through other connections. The first anomaly message is about the first connection being unavailable.

[0143] S904, the backup server of the primary server responds to the second connection establishment request sent by the client, establishes a second connection with the client, and the client resumes the transmission of the first data through the second connection.

[0144] In one possible implementation, the storage server cluster includes a master server and one or more backup servers of the master server; the method further includes: establishing a second connection between the backup servers and accessing the first data through the master server to access the storage medium where the first data is located; or establishing a second connection between the backup servers and bypassing the master server to access the first data by directly accessing the storage medium where the first data is located.

[0145] In one possible implementation, the method further includes: responding to an abnormal event in the server cluster subscribed to by the client by sending the abnormal event to the client via an abnormal message notification connection; the abnormal event types include master server failure, master server network failure, link failure, and link switching during operation and maintenance.

[0146] The embodiments of this application can be applied in disaster recovery environments, such as in a multi-site active-active architecture, where if one site suffers a disaster (such as a network failure or geographical disaster), the other site can still continue to provide services. Figure 10 This application provides a schematic diagram of a multi-site active-active system. For example... Figure 10 As shown, for example, data center 1 is site 1, and data center 2 is site 2, located in different locations. Each site deploys multiple NFS clients and NFS servers. Data center 1 contains NFS server 1 and NFS server 2. Data center 2 contains NFS server 3 and NFS server 4. NFS clients can be located outside or within either data center and can access any NFS server. NFS clients can also be configured for multipath access. Data center 1 and data center 2 can use SAN storage as a shared storage device, allowing them to access the same data.

[0147] Client access requests are forwarded to the NFS server in data center 1 or data center 2, for example, according to a load balancing mechanism.

[0148] NFS client 1 accesses NFS server 1. NFS client 1 mounts the shared directory on NFS server 1 to its local machine using a mount command. During this process, client 1 establishes the first connection with NFS server 1. Simultaneously, NFS client 1 establishes exception message notification connections with NFS server 1, NFS server 2, NFS server 3, and NFS server 4. NFS client 2 accesses NFS server 4. NFS client 2 mounts the shared directory on NFS server 4 to its local machine using a mount command. During this process, client 2 establishes the first connection with NFS server 4. Simultaneously, NFS client 2 establishes exception message notification connections with NFS server 1, NFS server 2, NFS server 3, and NFS server 4.

[0149] Thus, if the first connection from NFS client 1 to NFS server 1 becomes unavailable due to NFS server failure, network failure, or link failure, the system will select any one of NFS servers 2 to 4 to restore the first connection.

[0150] In a multi-site active-active architecture, network latency increases significantly when the client and server are located in different data centers. Due to the large physical distance between data centers, data packets need to pass through multiple routers and switches and may traverse different network areas, leading to increased transmission latency. During retrying, the timeout time and number of retries for the NFS client increase significantly, resulting in greater latency for the client while waiting for a server response. This application's embodiment addresses this by establishing an exception message notification connection, sending fault information from the NFS master server. The NFS client can quickly restore its connection to the data center, greatly shortening the time from the initial connection becoming unavailable due to an NFS client failure to the establishment of a second connection between the NFS client and the NFS server.

[0151] Based on the same concept as the method embodiments of this application, this application also provides a connection restoration device. The connection restoration device includes several modules, each module being used to execute various steps in the connection restoration method provided in the embodiments of this application. The division of modules is not limited here. Those skilled in the art will clearly understand that in practical applications, the various steps in the connection restoration method provided in the embodiments of this application can be assigned to different modules as needed, that is, the internal structure of the device can be divided into different modules to complete all or part of the functions described above. The modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more modules can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0152] For example, the connection restoration device is used to perform the connection restoration method provided in the embodiments of this application. Figure 11 This is a schematic diagram of the connection restoration device provided in an embodiment of this application. Figure 11 As shown, the connection restoration device provided in this application embodiment includes:

[0153] The establishment module 1101 is used for the client to establish a first connection with the master server of the storage server cluster; it is also used to establish a second connection with the backup server in the storage server cluster in response to receiving a first exception message sent by a backup server in the storage server cluster, wherein the first exception message relates to the first connection being unavailable, and the storage server cluster includes one or more backup servers; the transmission module 1102 is used for the client to perform a first data transmission with the server cluster through the first connection; the recovery module 1103 is used for the client to recover the transmission of the first data through the second connection.

[0154] Based on the same concept as the method embodiments of this application, this application also provides a connection restoration device. The connection restoration device includes several modules, each module being used to execute various steps in the connection restoration method provided in the embodiments of this application. The division of modules is not limited here. Those skilled in the art will clearly understand that in practical applications, the various steps in the connection restoration method provided in the embodiments of this application can be assigned to different modules as needed, that is, the internal structure of the device can be divided into different modules to complete all or part of the functions described above. The modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more modules can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0155] For example, the connection restoration device is used to perform the connection restoration method provided in the embodiments of this application. Figure 12 This is a schematic diagram of the connection restoration and transmission device provided in an embodiment of this application. Figure 12 As shown, the connection recovery device provided in this application embodiment is applied to a storage service cluster and includes: an establishment module 1201, configured to establish a first connection between the master server in the storage service cluster and the client in response to a first connection establishment request sent by the client; and further configured to send a first exception message to the client through another connection after detecting an anomaly in the first connection, the first exception message being about the unavailability of the first connection; a transmission module 1202, configured to provide the client with the transmission of first data through the first connection; and a recovery module 1203, configured to establish a second connection between the backup server of the master server and the client in response to a second connection establishment request sent by the client, and restore the transmission of the first data through the second connection.

[0156] Based on the same concept as the method embodiments of this application, embodiments of this application also provide a storage server. Figure 13 A storage server is provided as an embodiment of this application. For example... Figure 13 As shown, the storage server provided in this application embodiment includes a processor 1301, a memory 1302, and a communication interface 1303.

[0157] In this embodiment, the processor 1201 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0158] The memory 1202 may include a large-capacity memory for data or instructions, thereby providing storage space for the network device's operating system and executable program code, which may include, but is not limited to: Windows system (an operating system), Linux system (an operating system), HarmonyOS system (an operating system), etc., without limitation.

[0159] For example, and not as a limitation, memory 1302 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 1302 may include removable or non-removable (or fixed) media. Where appropriate, memory 1302 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 1302 is non-volatile solid-state memory.

[0160] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory may include one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software that may include computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to this application.

[0161] For example, a computer program may be stored on the memory 1302, and the processor 1301 executes the computer program to implement the steps in the above method embodiments. Alternatively, the processor 1201 executes the computer program to implement the functions of each module in the above device embodiments. Exemplarily, the computer program may be divided into one or more modules / units, which may be a series of computer program instruction segments capable of performing a specific function. The one or more modules / units are stored in the memory 1302 and executed by the processor 1301 to complete this application. For example, the computer program may be divided into multiple modules, as in the modules of the device described above.

[0162] The communication interface 1303 is used to send and receive data, for example, to send data processed by the processor 1301 to other network devices, or to receive data sent by other network devices.

[0163] Of course, for the sake of simplicity, Figure 13 Only some of the components of the storage server 1300 relevant to this application are shown, omitting components such as buses, input / output interfaces, etc. In addition, the storage server 1300 may include any other suitable components depending on the specific application. Furthermore, the computing device may be a desktop computer, laptop, handheld computer, cloud server, or other computing device. Those skilled in the art will understand that... Figure 13 This is merely an example of storage server 1300 and does not constitute a limitation on the computing device. It may include more or fewer components than illustrated, or combine certain components, or use different components. For example, the computing device may also include input devices, output devices, network access devices, buses, etc. For example, the input device may be a microphone array, and may also include, for example, a keyboard, mouse, etc. For example, the output device may output various information to the outside, and may include, for example, a monitor, speakers, printer, and communication networks and their connected remote output devices, etc.

[0164] This application also provides a storage server cluster. Figure 14 This application provides a storage server cluster 1400. For example... Figure 14 As shown, the storage server cluster includes at least one storage server 1300. The memory 1302 of one or more storage servers 1300 in the storage server cluster may store the same memory for executing... Figure 9The instructions for the method shown. The establishment module 1201, transmission module 1202, and recovery module 1203 in the connection recovery device can be deployed on one storage server 1300 or distributed across multiple storage servers 1300. In the case of distributed deployment, the establishment module 1201 can be deployed on one computing device or distributed across multiple storage servers 1300; similarly, the transmission module 1202 and recovery module 1203 can also be deployed on one storage server 1300 or distributed across multiple storage servers 1300.

[0165] The storage server may be a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0166] In some possible implementations, the memory 1302 of one or more storage servers 1300 in the storage server cluster may also store data for execution. Figure 9 The instructions for the method shown are partial. In other words, a combination of one or more storage servers 1300 can coexist to store instructions for executing... Figure 9 The instructions for the method shown.

[0167] It should be noted that the storage devices 1302 in the different storage services 1300 within the storage server cluster can store different instructions, each for execution. Figure 13 The illustrated device performs some of its functions. Specifically, the instructions stored in the memory 1302 of the different storage servers 1300 can implement... Figure 13 The function of one or more modules in the device shown.

[0168] In some possible implementations, one or more storage servers in a storage server cluster can be connected via a network. This network can be a wide area network (WAN), a local area network (LAN), or similar. Figure 14 One possible implementation is shown. For example... Figure 14 As shown, the two storage servers 1300A and 1300B are connected via a network. Specifically, they are connected to the network through the communication interfaces in each storage server. In this possible implementation, the memory 1302 in storage server 1300A stores instructions for implementing the functions of the setup module 1201. Simultaneously, the memory 1302 in storage server 1300B stores instructions for implementing the functions of the transmission module 1202 and the recovery module 1203.

[0169] It should be understood that Figure 14The functions of storage server 1300A shown can also be performed by multiple storage servers 1300. Similarly, the functions of storage server 1300B can also be performed by multiple storage servers 1300.

[0170] In addition to the methods, apparatus, and storage servers described above, embodiments of this application may also provide a computer program product, comprising computer program instructions. When executed by a processor, these computer program instructions cause the processor to perform the steps of the methods in the various embodiments of this application described in the "Method" section of this specification. The computer program product may be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The computer program code may be in source code form, object code form, executable file, or some intermediate form. The computer program code may be executed entirely on a user's computing device, partially on a user's device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0171] Furthermore, embodiments of this application may also provide a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps of the display control method according to various embodiments of this disclosure as described in the "Method" section above. The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. It should be noted that the content contained in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, a computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0172] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0173] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application. It should be understood that in the embodiments of this application, the order of the process numbers does not imply the order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0174] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A connection restoration method, characterized in that, The method includes: The client establishes the first connection with the master server of the storage server cluster; The client performs a first data transmission with the server cluster through the first connection; In response to receiving a first exception message sent by a backup server in the storage server cluster, the client establishes a second connection with the backup server in the storage server cluster, wherein the first exception message relates to the first connection being unavailable, and the storage server cluster includes one or more backup servers; The client resumes the transmission of the first data through the second connection.

2. The method according to claim 1, characterized in that, The method further includes: The backup server that establishes the second connection accesses the first data through the primary server by accessing the storage medium containing the first data; or The backup server that establishes the second connection bypasses the primary server and accesses the first data by directly accessing the storage medium where the first data is located.

3. The method according to any one of claims 1 or 2, characterized in that, The method further includes: The backup server stores backup data of the primary server. After the client establishes a second connection with the backup server in the storage server cluster, the transmission of the first data is restored based on the backup data; or After the client establishes a second connection with the backup server in the storage server cluster, the backup server accesses the storage medium where the first data is located and resumes the transmission of the first data.

4. The method according to claim 1, further comprising: The backup server detects an anomaly in the primary server. If an anomaly is detected in the primary server, the backup server sends a first anomaly message to the client.

5. The method according to any one of claims 1 to 4, characterized in that, The establishment of the second connection includes: the backup server that establishes the second connection with the client is bound to a virtual IP, and the virtual IP was originally the virtual IP used by the primary server and the client to communicate, and the client and the backup server establish the second connection through the virtual IP.

6. The method according to claim 1, characterized in that, The method further includes: configuring multi-path access between the client and the storage server cluster, wherein the multi-path access includes: the paths between the client and each of the multiple storage servers in the storage server cluster that have backup relationships; The client establishes a second connection with a backup server in the storage server cluster, including establishing an available path in the multipath access as the second connection.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: establishing an exception message notification connection between the client and the backup server in the storage server cluster; The first error message is sent via the error message notification connection.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: after the client obtains the file lock corresponding to the first data, it performs first data transmission with the storage cluster through the first connection; The client resumes the transmission of the first data through the second connection, including: The client responds upon receiving the file lock sent by the backup server; The transmission of the first data is resumed through the second connection.

9. The method according to any one of claims 1-8, characterized in that, The method further includes: The client subscribes to abnormal events in the server cluster; the types of abnormal events include master server failure, master server network failure, link failure, and link switching during operation and maintenance.

10. A connection restoration method, characterized in that, Applied to a storage service cluster, the method includes: In response to the first connection establishment request sent by the client, the master server in the storage service cluster establishes a first connection with the client. The master server provides the client with the transmission of first data through the first connection; After a backup server that has a backup relationship with the primary server in the storage service cluster detects that the first connection is abnormal, it sends a first abnormal message to the client through other connections. The first abnormal message is about the first connection being unavailable. In response to the second connection establishment request sent by the client, the backup server of the primary server establishes a second connection with the client and resumes the transmission of the first data through the second connection.

11. The method according to claim 10, characterized in that, The storage server cluster includes a master server and one or more backup servers for the master server; the method further includes: The backup server establishing the second connection accesses the first data file through the primary server by accessing the storage medium containing the first data; or The backup server establishes the second connection, bypassing the primary server, and accesses the first data file directly by accessing the storage medium where the first data is located.

12. The method according to any one of claims 10 or 11, characterized in that, After establishing the first connection with the client, the method further includes: In response to the exception message notification establishment request sent by the client, an exception message notification connection is established with the client; the first exception message is sent through the exception message notification connection.

13. The method according to claim 12, characterized in that, The method further includes: In response to an abnormal event in the server cluster subscribed to by the client, the abnormal event is sent to the client via the abnormal message notification connection; the abnormal event types include master server failure, master server network failure, link failure, and link switching during operation and maintenance.

14. A connection restoration device, characterized in that, The device includes: The module establishes a first connection between the client and the master server of the storage server cluster; it is also used to establish a second connection between the client and the backup server in the storage server cluster in response to receiving a first exception message sent by a backup server in the storage server cluster, wherein the first exception message relates to the first connection being unavailable, and the storage server cluster includes one or more backup servers. The transmission module is used for the client to perform first data transmission with the server cluster through the first connection; A recovery module is used by the client to resume the transmission of the first data through the second connection.

15. A connection restoration device, characterized in that, The device is used in a storage service cluster and includes: The module is configured to establish a first connection with the client in response to a first connection establishment request sent by the client; it is also configured to send a first exception message to the client via another connection after detecting an exception in the first connection, the first exception message being related to the first connection being unavailable. The transmission module is used to provide the client with the transmission of first data through the first connection; The recovery module is used to respond to a second connection establishment request sent by the client, and the backup server of the primary server establishes a second connection with the client to restore the transmission of the first data through the second connection.

16. A storage server, characterized in that, include: Memory, used to store executable code; A processor, configured to implement the method of any one of claims 10-13 when executing the executable code.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed in a computer, causes the computer to perform the method according to any one of claims 1-9, or the method according to any one of claims 10-13.

18. A computer program product comprising, characterized in that, When the method is run on a computer, it causes the computer to perform the method according to any one of claims 1-9, or the method according to any one of claims 10-13.

Images