Virtual sockets for load balancing and failover

Abstract

The present disclosure relates to virtual sockets for load balancing and failover. One aspect of the present application can provide a system and method for balancing load among a plurality of network sockets established between a local node and a remote node. During operation, the system can encapsulate the plurality of network sockets to form a local transport layer virtual socket comprising a write interface and a read interface. The system can receive a packet at the write interface of the local transport layer virtual socket; select a network socket from the plurality of network sockets based on a load balancing policy; and forward the packet to a socket-specific incoming queue associated with the selected network socket to allow the packet to be sent to a corresponding read interface of a remote transport layer virtual socket via the selected network socket.

Description

Background Technology

[0001] This disclosure generally relates to network load balancing and fault tolerance. Attached Figure Description

[0002] Figure 1 An exemplary networking system including a client and a server according to one aspect of this application is illustrated.

[0003] Figure 2 An exemplary architecture of a virtual socket according to one aspect of this application is illustrated.

[0004] Figure 3A An exemplary client / server system implementing virtual sockets according to one aspect is illustrated.

[0005] Figure 3B The illustration shows an exemplary virtual socket pair according to one aspect.

[0006] Figure 4 An exemplary block diagram is shown of a client device that facilitates load balancing and failover based on virtual sockets, according to one aspect.

[0007] Figure 5 A flowchart illustrating an exemplary packet switching process in a client / server system implementing virtual sockets according to one aspect of this application is presented.

[0008] Figure 6 A flowchart illustrating an exemplary failover operation in a client / server system implementing virtual sockets according to one aspect of this application is presented.

[0009] Figure 7 An exemplary computer system facilitating virtual sockets in a client / server system according to one aspect of this application is illustrated.

[0010] In these accompanying drawings, the same reference numerals refer to the same elements. Detailed Implementation

[0011] The following description is presented to enable any person skilled in the art to make and use the examples, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed examples will be apparent to those skilled in the art, and the general principles defined herein can be applied to other examples and applications without departing from the spirit and scope of this disclosure. Therefore, the scope of this disclosure is not limited to the examples shown, but is intended to be consistent with the maximum scope of the principles and features disclosed herein.

[0012] In client / server networks (e.g., storage network systems) where clients and servers interconnect using transport layer sockets (e.g., Transmission Control Protocol (TCP) sockets), load balancing and failover are critical to system performance and reliability. Load imbalance can lead to network congestion, while link failures can result in traffic loss. It is desirable to provide a load balancing and failover mechanism that fully utilizes available resources in the system to maximize TCP throughput and ensures fault tolerance in connections.

[0013] Some client devices (e.g., clients in a storage network with one or more storage nodes) may include multiple network interfaces (e.g., a client device may be equipped with multiple network interface cards (NICs)) and may establish one or more TCP sockets via each network interface. To achieve high throughput and high availability, it is desirable to distribute traffic load across multiple TCP sockets and allow for failover in the event of interface failure. However, traditional load balancing solutions often attempt to balance the load across multiple servers without addressing the need to balance the load across TCP sockets. Highly congested network sockets may delay packet forwarding from the client to the server, and vice versa. Similarly, traditional failover solutions for link failures do not provide failover between TCP sockets. For example, a failed TCP socket / interface may result in packet loss. Currently, there is no solution that achieves load balancing and failover across multiple TCP sockets. Furthermore, because multiple threads can execute concurrently on the client or server, the load balancing / failover solution needs to be thread-safe.

[0014] According to some aspects of this application, to facilitate load balancing and failover among TCP sockets, virtual sockets can be constructed by encapsulating multiple TCP sockets. Applications running on the client or server can treat the virtual socket as a single socket with a single write interface and a single read interface, respectively for transmitting and receiving packets. The virtual socket can include multiple incoming and outgoing queues for queuing packets arriving at and departing from the virtual socket, respectively. On the client or server side, each encapsulated TCP socket can be communicatively coupled to a single socket-specific incoming queue and multiple thread-specific outgoing queues. Both the incoming and outgoing queues are thread-safe to ensure that packets in the queues can be accessed by multiple threads in the correct order. Load balancing among the encapsulated TCP sockets can be achieved by forwarding packets arriving at the virtual socket's write interface to a specific incoming queue based on the TCP socket's load. Furthermore, faulty TCP sockets can be removed from the virtual socket without affecting outgoing traffic. Additionally, when a faulty TCP socket recovers from a failure, it can be added back to the virtual socket. Virtual sockets can be transparent to applications running on the client and server, and do not add any extra burden to application developers in creating applications that can benefit from the high throughput provided by virtual sockets.

[0015] Figure 1 An exemplary networking system including a client and a server is illustrated according to one aspect. Networking system 100 may include a client 102 and a server 104. Client 102 may execute multiple client threads (e.g., threads 106, 108, and 110); and server 104 may execute multiple server threads corresponding to the client threads (e.g., threads 112, 114, and 116). More specifically, for each client thread executing on client 102, there is a specific, corresponding server thread executing on server 104. Client 102 may include multiple network interfaces (e.g., interfaces 118 and 120). Server 104 may include network interface 122.

[0016] When client threads 106, 108, and 110 are executed, client 102 can communicate with (e.g., read from or write to) the corresponding server thread executing on server 104. For example, client thread 106 can communicate with the corresponding server thread 112. To facilitate this communication, a TCP connection can be established between client 102 and server 104. For example, a TCP connection 124 can be established between network interface 118 of client 102 and network interface 122 of server 104, and a TCP connection 126 can be established between network interface 120 and network interface 122. The TCP connection between the client and server can be established by creating a pair of TCP sockets (each TCP socket is a connection-oriented software interface provided by the operating system that implements the TCP protocol) (one on the client side and one on the server side). The TCP connection can specify an end-to-end communication channel through which packets are transmitted between client 102 and server 104.

[0017] like Figure 1 As shown, each of the multiple client threads executing on client 102 can communicate with the corresponding server thread executing on server 104 via a TCP connection. For example, client thread 106 can communicate with the corresponding server thread 112 via TCP connection 124 or TCP connection 126. At any given time, different TCP connections may have different loads. Packets transmitted via a heavily loaded TCP connection may take longer to reach their destination. To ensure high throughput, a mechanism is desired that packets can be forwarded via different TCP connections based on the load of each corresponding TCP connection. Similarly, when a hardware failure occurs on a network interface, the corresponding TCP connection may become unavailable, and it is then desirable to forward packets via other available TCP connections. For example, network interface 118 on client 102 may experience a hardware failure, making TCP connection 124 unavailable. In this case, all packets from client 102 can be forwarded via TCP connection 126 to prevent packet loss. It should be noted that... Figure 1 It is a high-level conceptual diagram that only shows transport layer (i.e., TCP) communication. Although Figure 1 It is not shown in the diagram, but each TCP connection may include a pair of TCP sockets, one client socket on client 102 and one server socket on server 104.

[0018] According to some aspects of this disclosure, virtual socket pairs can be established to include multiple TCP socket pairs between the client and the server, thereby achieving load balancing and failover among these multiple TCP socket pairs. More specifically, to ensure thread safety (i.e., to avoid data contention caused by different threads accessing shared data simultaneously), thread-safe queues can be used to queue incoming and outgoing packets. Because sockets always exist in pairs, and each socket pair defines a unique connection between the client and the server, in this disclosure, a socket pair (virtual socket pair or traditional TCP socket pair) may sometimes be referred to as a socket or a connection.

[0019] Figure 2 An exemplary architecture for a virtual socket according to one aspect is illustrated. Virtual socket 200 can be constructed by grouping multiple transport layer sockets (e.g., TCP sockets 202 and 204) and providing multiple queues for these TCP sockets. It should be noted that... Figure 2 Each TCP socket shown represents a TCP connection between the client and the server (similar to...). Figure 1 The diagram shows a pair of TCP sockets corresponding to TCP connections 124 or 126. Therefore, virtual socket 200 can correspond to multiple TCP connections between the client and server. These queues can be used to multiplex network connections and packets. More specifically, on both the client and server sides, each TCP socket can be communicatively coupled to an incoming queue (which queues packets entering the TCP socket) and multiple outgoing queues (which queue packets leaving the TCP socket). The terms "incoming queue" and "outgoing queue" are defined for TCP sockets. Incoming queues can be socket-specific, meaning each incoming queue is dedicated to a specific TCP socket, while outgoing queues can be shared among multiple TCP sockets. To ensure thread safety, all incoming and outgoing queues are thread-safe, meaning these queues can be accessed safely by multiple threads (e.g., data is dequeued sequentially for each thread) and thread synchronization can be maintained.

[0020] exist Figure 2 In the example shown, on the client side, TCP socket 202 can be coupled to incoming queue 206, while TCP socket 204 can be coupled to incoming queue 208. Thus, if a packet is to be transmitted through a specific TCP socket or connection, the packet will be queued in the incoming queue corresponding to that specific TCP socket. Figure 2It is also shown that, on the client side, each outgoing queue (e.g., outgoing queues 210, 212, and 214) can be coupled to both sockets 202 and 204. Similarly, on the server side, TCP socket 202 can be coupled to incoming queue 216, TCP socket 204 can be coupled to incoming queue 218, and sockets 202 and 204 can each be coupled to multiple outgoing queues (e.g., outgoing queues 220, 222, and 224). Depending on some aspects, outgoing queues can be thread-specific, meaning that packets originating from a specific thread executing on the client or server will be queued in an outgoing queue dedicated to the corresponding destination thread. Each outgoing queue can be shared by all TCP sockets in the virtual socket. Figure 2 In this example, assume there are three threads executing on each of the client and server sides. Therefore, there are three outgoing queues on each side (client and server). It should be noted that although represented as a single entity, Figure 2 Each TCP socket shown is essentially a pair of sockets: one established on the client side and one established on the server side. From the client's perspective, the client-side socket can be called a local socket, while the server-side socket can be called a remote socket. Similarly, from the server's perspective, the server-side socket is a local socket, while the client-side socket is a remote socket.

[0021] like Figure 2 As shown, virtual socket 200 provides multiple paths between the client and the server. Each path may include a TCP socket, an incoming queue for multiplexing packets from different threads, and an outgoing queue for multiplexing packets from different paths. For example, Figure 2 The illustration shows that in each direction, a given packet flow can take one of two inbound queues and one of three outbound queues, resulting in six possible paths. The number of inbound queues depends on the number of TCP sockets included in the virtual sockets, while the number of outbound queues depends on the number of threads executing on the client and server. There can be more inbound queues than outbound queues, and vice versa. An exemplary client-to-server path may include inbound queue 206, TCP socket 202, and outbound queue 220; an exemplary server-to-client path may include inbound queue 218, TCP socket 204, and outbound queue 214. Figure 2 In the example shown, it is assumed that there are three threads, so each TCP socket can provide three paths, each path is used for packets belonging to a specific thread, and each path consists of an incoming queue, a TCP socket, and an outgoing queue.

[0022] Load balancing between TCP sockets 202 and 204 can be achieved by propagating incoming packets across multiple paths. For example, a client or server can select a TCP socket (and thus one of several paths associated with TCP) for packet transmission based on the socket's load. The actual path chosen also depends on the originating thread of the incoming packets. Depending on several factors, a client or server can select the socket with the lowest load (e.g., the socket with the fewest packets in its incoming queue) for packet transmission. The load on each socket can be determined based on the occupancy of the corresponding incoming queue. For example, when a client transmits packets to a server, the client (e.g., write interface logic on the client) can determine whether to send packets to TCP sockets 202 or 204 based on the occupancy of incoming queues 206 and 208. Queue occupancy can be calculated based on the queue's capacity (i.e., the amount of allocated memory space) and the current amount of data in the queue. Other load balancing techniques (e.g., round-robin or hash-based) can also be used to balance the load between TCP sockets 202 and 204. Similarly, when the server transmits packets to the client, it can determine whether to send the packets to TCP sockets 202 or 204 based on the relative occupancy of incoming queues 216 and 218. Although shown as a single entity, virtual socket 200 actually represents a pair of virtual sockets: a client-side virtual socket and a server-side virtual socket. (See reference...) Figure 4 The specific implementation of the internal logic unit used to implement client-side virtual sockets or server-side virtual sockets is further described.

[0023] Virtual socket 200 can also enable failover between TCP sockets 202 and 204. When one of the TCP sockets fails (e.g., due to a hardware failure on the client-side or server-side network interface), it can be removed from virtual socket 200, so that all packets entering virtual socket 200 will be transmitted via the remaining TCP sockets. For example, in the event of a failure in TCP socket 204, if the client intends to transmit a packet to the server, it will send the packet to the incoming queue 206.

[0024] When implemented in a client / server system, virtual sockets (or pairs of virtual sockets) provide an end-to-end communication channel for transmitting packets between the client and server. When multiple threads are running simultaneously on both the client and server, virtual sockets can provide an end-to-end communication channel for each thread. Figure 3AAn exemplary client / server system implementing virtual sockets according to one aspect is illustrated. Client / server system 300 may include client 302 and server 304. Client 302 may execute multiple client threads (e.g., threads T1, T2, and T3), and server 304 may execute corresponding server threads. Threads on client 302 may communicate with corresponding threads on server 304, i.e., send packets to and receive packets from the corresponding threads on the server. For example, thread T1 on client 302 may send packets to and receive packets from thread T1 on server 304. More specifically, packets may be exchanged via virtual socket 306. As previously stated, for simplicity, virtual socket 306 is shown as a single entity. In practice, virtual socket 306 may be a pair of virtual sockets comprising a client-side virtual socket residing on client 302 and a server-side virtual socket residing on server 304.

[0025] Virtual socket 306 can connect to client and server interfaces via a pair of interfaces on either side (client side or server side). These interfaces allow the client or server to send and receive packets. Threads executing on client 302 or server 304 can only detect this pair of interfaces, thus treating virtual socket 306 as a single socket / connection. On the client side, virtual socket 306 may include a write interface (WI) 308 and a read interface (RI) 310. Threads on client 302 can send packets to write interface 308 and receive packets from read interface 310. Similarly, on the server side, virtual socket 306 may include a write interface (WI) 312 and a read interface (RI) 314. Threads on server 304 can send packets to write interface 312 and receive packets from read interface 314.

[0026] Virtual socket 306 can encapsulate multiple TCP sockets, such as TCP sockets 316 and 318. It should be noted that encapsulating TCP sockets means that the TCP sockets are hidden from both the client and server and can only be accessed via the interfaces created for the virtual socket (i.e., the write interface and the read interface). Because virtual socket 306 is shared by multiple threads on client 302, thread synchronization mechanisms can be used to ensure that packets from different threads are not out of order. Traditional methods rely on thread locks to synchronize threads, which can be cumbersome and inefficient. Depending on several aspects, lock-free thread synchronization schemes can be used to ensure efficiency and high performance. More specifically, thread-safe multi-producer single-consumer queues can be used for packet and / or connection multiplexing instead of using locking mechanisms to implement thread safety. Thread-safe queues allow multiple threads to access the queue without affecting each other's execution. Various techniques can be used to implement thread-safe queues, and the scope of this disclosure is not limited to the actual techniques used to implement thread-safe queues.

[0027] Depending on several aspects, in the packet inbound direction (the direction in which packets enter virtual socket 306), multiple threads can be producers of the inbound queues and the TCP socket can be a single consumer of the inbound queues. For example, on the client side, write interface 308 can forward packets from all threads on client 302 (e.g., threads T1, T2, and T3) to one of the inbound queues (e.g., inbound queue 320 or 322). Each inbound queue can be dedicated to a single TCP socket, meaning that packets in that queue will only be sent to the coupled TCP socket. Figure 3A In the example shown, incoming queue 320 is dedicated to TCP socket 316, while incoming queue 322 is dedicated to TCP socket 318. Load balancing logic unit (not in...) Figure 3A As shown in, and referenced Figure 4 (Further description) It can be configured to determine which TCP socket to forward incoming packets to based on the load of each individual TCP socket. The load of each TCP socket can be related to the number of packets in the corresponding incoming queue. According to one aspect, incoming packets should be forwarded to the TCP socket with the least load. To this end, the load balancing logic unit can determine the number of packets in each incoming queue and select the incoming queue with the fewest packets. Alternatively, the load balancing logic unit can select the incoming queue in a round-robin manner or based on a hash function.

[0028] In addition, the failover logic unit ( Figure 3A Not shown in the image, and refer to [reference]. Figure 4(Further description) can be configured to facilitate failover between TCP sockets. In response to determining a hardware failure on one of the network interfaces, the failover logic unit can disable the corresponding TCP socket (or remove the TCP socket from virtual socket 306). Thus, write interface 308 no longer forwards incoming packets to the incoming queue corresponding to the disabled TCP socket. When a TCP socket is disabled, the corresponding incoming queue is also disabled, and existing packets in the disabled incoming queue can be forwarded to the incoming queues of other active TCP sockets. After the network interface recovers from the failure, a new TCP socket and its corresponding incoming queue can be established and added to virtual socket 306. Server-side incoming queues (e.g., queues 324 and 326) can operate similarly. Write interface 312 can forward packets from all threads on server 304 (e.g., threads T1, T2, and T3) to one of the server-side incoming queues. The server-side load balancing logic unit of virtual socket 306 can balance the load between TCP sockets by selecting the server-side incoming queue, and the server-side failover logic of virtual socket 306 handles failover in response to interface failure on server 304.

[0029] Depending on several aspects, in the outgoing direction of the packet (the direction in which the packet leaves virtual socket 306), multiple sockets can be producers of outgoing queues and threads can be single consumers of outgoing queues. For example, on the server side, each TCP socket can send outgoing packets to one of the outgoing queues (e.g., outgoing queues 328, 330, and 332) based on the destination thread. Outgoing packets destined for thread T1 can be sent to outgoing queue 322, which is accessible only to thread T1 on server 304. Similarly, outgoing queue 324 is accessible only to thread T2, and outgoing queue 326 is accessible only to thread T3. Once an outgoing packet is queued in a particular outgoing queue, a notification can be sent to the corresponding thread to prompt it to read the outgoing queue to obtain the packet. For example, when a packet arrives at outgoing queue 328, thread T1 on server 304 can receive a notification and subsequently read the packet via read interface 314. The operation of outgoing queues on the client side (e.g., outgoing queues 334, 336, and 338) can be similar, such that packets from server 304 can be queued in those outgoing queues before being read by the corresponding thread on client 302 via read interface 310.

[0030] It should be noted that during failover, although the incoming queues of the corresponding disabled TCP sockets may also be disabled, the outgoing queues are unaffected because they continue to receive packets from the remaining active TCP sockets.

[0031] exist Figure 3A In the example shown, a transport layer virtual socket is established between client 302 and server 304. In reality, the number of virtual sockets between the client and server is configurable. For example, multiple network sockets / connections established between the client and server can be grouped into a single virtual socket or multiple virtual sockets depending on memory limitations. A larger number of virtual sockets may require the client or server devices to have larger memory to support the increased number of queues. Load balancing and failover can be implemented among multiple TCP sockets within each virtual socket. Besides connecting independent client and server devices, virtual sockets can also be used to connect client clusters and server clusters. More specifically, one or more virtual sockets can be established between a client cluster and a server cluster. Furthermore, in Figure 3A In the example shown, each network socket is assigned an incoming queue in each direction. In practice, each incoming queue can include multiple priority sub-queues to achieve priority-based load balancing.

[0032] As mentioned above, Figure 3A The virtual socket 306 shown represents a pair of virtual sockets, one residing on the client device and the other on the server. Depending on the perspective, each socket can be considered a local socket or a remote socket. Figure 3B An exemplary virtual socket pair according to one aspect is illustrated. Virtual socket pair 340 may include a local virtual socket 342 and a remote virtual socket 344. Local virtual socket 342 may be established by encapsulating local TCP sockets 346 and 348, while remote virtual socket 344 may be established by encapsulating corresponding remote TCP sockets 350 and 352. Each virtual socket may include a write interface for receiving packets from threads running on a corresponding network node (which may be a client or a server), and a read interface for sending packets to these threads. For example, local virtual socket 342 may include a write interface 354 and a read interface 356. Packets received via the write interface may be queued in a socket-specific incoming queue, while packets sent via the read interface may be queued in a thread-specific outgoing queue.

[0033] At local virtual socket 342, incoming packets can be queued by selecting a queue-specific incoming queue (e.g., incoming queue 358 or 360) via write interface 354, thereby achieving load balancing. More specifically, the incoming queue can be selected based on the load of the corresponding local TCP socket. If a particular local TCP socket (e.g., local TCP socket 346 or 348) has a lighter load than other local TCP sockets, the incoming queue serving that particular TCP socket will be selected to queue the incoming packets. The packet can then be sent to the remote node via the local TCP socket corresponding to the selected incoming queue. At the remote node, the packet can be received via the corresponding remote TCP socket. Similarly, the local node can receive packets from the remote node via one of the local TCP sockets. Depending on the destination thread of the received packet, the received packet can be sent to one of the outgoing queues 362, 364, and 366. The destination thread can then read the packet from the outgoing queue via read interface 356.

[0034] Figure 4 An exemplary block diagram of a client device facilitating virtual socket-based load balancing and failover is illustrated according to one aspect. Client device 400 may include multiple processor cores (e.g., cores 402, 404, and 406), each executing one or more threads. According to one aspect, each core may execute one thread. For example, core 402 may execute thread T1, core 404 may execute thread T2, and core 406 may execute thread T3. Other arrangements are also possible. For example, not all cores may be active, or a single core may execute multiple threads. According to another aspect, client device 400 may be part of a storage system, and the threads may be data path threads (e.g., threads for data transfer), which are typically single threads running on each core, one thread per core. Because the number of cores on each client device is limited, the number of active threads can be configurable and small. For example, a client device may include up to eight cores, and the number of cores that can be activated (and therefore the number of threads) can be user-configurable.

[0035] Client device 400 may include a write interface 408 and a read interface 410. Each interface may be an application programming interface (API) that allows client device 400 to exchange data with other devices or computer programs. For example, write interface 408 may allow client device 400 to send packets to other devices (e.g., servers) or programs, and read interface 410 may allow client device 400 to receive packets from other devices or programs. For example, a thread executing on one of the cores on client device 400 can send packets to a corresponding thread executing on a server device by sending (or writing) packets to write interface 408. Similarly, a thread executing on one of the cores on client device 400 can receive packets from a corresponding thread executing on a server device by receiving (or reading) packets from read interface 410. Both write interface 408 and read interface 410 may be thread-safe, meaning that multiple threads can access these interfaces without the risk of thread conflicts.

[0036] Client device 400 may include multiple network interfaces, such as interfaces 412 and 414. Each network interface may be associated with an Internet Protocol (IP) address. In one example, each network interface may include a network interface card (NIC). Client device 400 can send and receive packets via the network interfaces. When sending and receiving packets, client device 400 may create a transport layer network socket (e.g., a TCP socket) for each network interface. Figure 4 In the example shown, network socket 416 corresponds to network interface 412, and network socket 418 corresponds to network interface 414. It should be noted that a network interface is the physical interface for sending and receiving packets, and a network socket defines a logical interface or endpoint for a process executing on client device 400 to send packets to and receive packets from a remote process.

[0037] Client device 400 may include multiple logical units, such as logical units 420-428, for maintaining queues. Packets may be queued in an incoming queue before being sent via a network socket; and in an outgoing queue before being received by a process executing on one of the cores. The terms "incoming" and "outgoing" are chosen in relation to the network socket. As previously mentioned, each incoming queue is dedicated to one network socket, while the outgoing queue is shared by all network sockets. For example, incoming queue logical unit 420 may be responsible for maintaining an incoming queue dedicated to network socket 416, while incoming queue logical unit 422 may be responsible for maintaining an incoming queue dedicated to network socket 418. Although outgoing queues are shared by all sockets, each outgoing queue may be dedicated to a specific thread. In other words, a thread can only receive (or read) packets from its corresponding outgoing queue. Figure 4 In the example shown, outgoing queue logic units 424, 426, and 428 can be responsible for maintaining outgoing queues dedicated to threads T1, T2, and T4, respectively. Client device 400 may include memory 440, which can provide physical resources to be allocated to the various incoming and outgoing queues.

[0038] Client device 400 may include a load balancing logic unit 432 configured to implement a load balancing strategy to balance the load between network sockets 416 and 418. According to one aspect, load balancing logic unit 432 may be configured to detect the load on each network socket and select the socket with the lowest load to forward packets from threads executing on one of the cores. For example, load balancing logic unit 432 may determine the occupancy of each incoming queue and select the incoming queue with the lowest occupancy. The queue occupancy can be determined by dividing the amount of data in the occupied queue by the total amount of memory allocated to that queue. Alternatively, load balancing logic unit 432 may determine the number of packets queued in each incoming queue and select the incoming queue with the fewest queued packets. Load balancing logic unit 432 may then send a control signal to write interface 408, instructing write interface 408 to forward packets to the selected incoming queue. The queued packets can then be transmitted to the server device via the corresponding network socket. Other load balancing strategies are also possible. For example, incoming packets can be sent to different sockets / incoming queues either in a round-robin fashion or by calculating a hash function (e.g., based on the packet header).

[0039] Client device 400 may further include a failover logic unit 434 configured to facilitate failover between network sockets 416 and 418. According to one aspect, the failover logic unit 434 may be configured to detect a hardware fault on a network interface, and in response to detecting a faulty network interface, the failover logic unit 434 may be configured to disable the corresponding network socket. For example, in response to detecting a fault on a network interface, the failover logic unit 434 may send a signal to write interface 408 to instruct write interface 408 to redirect packets to stop forwarding packets to the incoming queue corresponding to the disabled socket. Figure 4In the example shown, if the failover logic unit 434 detects a hardware failure on network interface 412, it can disable network socket 416 by sending a control signal to write interface 408 to instruct it to redirect incoming packets initially intended for forwarding to disabled network socket 416 to the remaining network sockets 418. More specifically, packets initially intended for incoming queue 420 are now redirected to incoming queue 422. Furthermore, packets currently queued in incoming queue 420 can be redistributed to incoming queue 422. Once the failed network interface recovers from the hardware failure, the failover logic unit 434 can restore the failed network socket by sending a control signal to write interface 408.

[0040] Network sockets, various incoming and outgoing queues, load balancing logic unit 432, and failover logic unit 434 together form client-side virtual socket 436. The internal structure (i.e., queues and network sockets) and control (e.g., load balancing and failover control) of virtual socket 436 are transparent to processes or applications executing on the individual cores of client device 400. More specifically, processes or applications only see a single socket (i.e., virtual socket 346) used for sending and receiving packets. Write interface 408 and read interface 410 are the incoming and outgoing interfaces of client-side virtual socket 436, respectively. Processes or applications can send and receive packets via write interface 408 and read interface 410, respectively.

[0041] In addition to allowing the server to establish multiple network sockets for a single network interface, the server can have an architecture similar to client 400. In a client / server system, network sockets established on the client device can have corresponding network sockets established on the server, thus forming network socket pairs. Similarly, client-side virtual socket 436 and its corresponding server-side virtual socket can form virtual socket pairs. The server-side virtual socket can have multiple logical units similar to those on the client side, including network sockets similar to network sockets 416 and 418, incoming queue logical units similar to incoming queue logical units 420-422, outgoing queue logical units similar to outgoing queue logical units 424-428, load balancing logical units similar to load balancing logical unit 432, and failover logical units similar to failover logical unit 434.

[0042] Figure 5A flowchart illustrating an exemplary packet switching process in a client / server system implementing virtual sockets according to one aspect is presented. This example assumes packets are transmitted from a client to a server. During operation, a thread executing on the client device can generate packets to be sent to a corresponding thread executing on the server (operation 502). It should be noted that although the client device may have multiple network interfaces (e.g., NICs) (meaning multiple network sockets can be established to allow the client device to communicate with the server), the thread is only aware of a single connection between the client device and the server (i.e., the virtual socket). This allows application developers to develop applications that can benefit from the load balancing and failover provided by virtual sockets without modifying their existing code. Multiple network sockets can be encapsulated within a virtual socket. A virtual socket can include a pair of virtual sockets, comprising a client-side virtual socket and a server-side virtual socket. The client-side virtual socket can include a write interface for sending packets and a read interface for receiving packets.

[0043] The generated packets can be sent to the write interface of the client-side virtual socket (Operation 504). Load balancing logic unit (e.g., Figure 4 The load balancing logic unit 432 shown can select network sockets encapsulated within virtual sockets to transmit packets (operation 506). According to some aspects, the load balancing logic unit can select network sockets based on the load of each individual network socket encapsulated within the virtual socket. Each network socket is coupled to a socket-specific incoming queue and can be configured to receive packets only from said socket-specific incoming queue. Determining the load of each individual network socket may involve determining the occupancy rate of each corresponding incoming queue or the number of packets in each queue. The load balancing logic can select the network socket with the minimum load. Alternatively, the load balancing logic can use other criteria to select network sockets, such as round-robin or hash-based criteria. The load balancing logic can send a load balancing instruction to the write interface to indicate the selected network socket (operation 508). The write interface can then forward packets to the socket-specific incoming queue corresponding to the network socket (operation 510). Figure 4 As an example, load balancing logic unit 432 can send a load balancing instruction to write interface 408 to notify write interface 408 that network socket 416 has been selected based on a predetermined load balancing strategy. Write interface 408 can then forward incoming packets to incoming queue logic unit 420 serving network socket 416.

[0044] The packet can then be sent to the server via the selected network socket (operation 512). It should be noted that the network socket actually includes both a client-side socket and a server-side socket, and the packet is transmitted by the client-side socket and received at the server-side socket. The physical network interface (e.g., NIC) bound to the client-side socket is responsible for transmitting the packet to the communication network coupling the client device and the server device. The physical network interface (e.g., NIC) bound to the server-side socket is responsible for receiving the packet. Various communication protocols can be used for packet transmission. According to some aspects, the client device and the server device communicate with each other using TCP, and the network socket is a TCP socket. It should be noted that this disclosure primarily relates to end-to-end communication between the client and the server (i.e., the transport layer), and other layers of the network (e.g., the network layer and the physical layer) are beyond the scope of this disclosure.

[0045] Once the server-side network socket receives the packet, it determines the destination thread of the packet (operation 514). According to the client / server model, the thread executing on the client device communicates with the corresponding thread executing on the server. Therefore, the server-side network socket can determine the destination thread of the packet based on the identity of the originating thread. Depending on some aspects, information about the originating thread may be included in the packet header. Based on the identity of the destination thread (e.g., by reading the packet header), the server-side network socket can forward the received packet to the corresponding thread-specific outgoing queue (operation 516). It should be noted that the number of threads executing on the client device and the server can be predefined to allow users to allocate resources for various queues in the virtual socket. In the case of a storage system, the number of threads is typically equal to the number of active processor cores, with one thread per active core. The server-side network socket can also send notifications to the server thread (operation 518). For example, the server-side network socket can read the packet header, determine the destination thread based on the packet header, and send a notification to the destination thread. Upon receiving a notification, the server thread reads the packet from the outgoing queue via the virtual socket's read interface (Operation 520).

[0046] The system's operation in the other direction (i.e., transmitting packets from the server to the client device) can be similar to Figure 5 The operations shown in the figure, except Figure 5The various client-side processes illustrated occur on the server, and vice versa. For example, a thread on the server can generate packets and send them to the server-side write interface of a virtual socket, and server-side load balancing logic can select a network socket encapsulated in the virtual socket to transmit the packets. The server-side write interface can forward the packets to the corresponding socket-specific inbound queue. The selected server-side socket can then dequeue the packets and send them to the corresponding client-side socket, which can then determine the destination thread and queue the packets in the corresponding thread-specific outbound queue. The server-side socket can also notify the server-side thread of the packets, and the server-side thread can then read the packets from the outbound queue via the server-side read interface of the virtual socket.

[0047] Figure 6 A flowchart illustrating an exemplary failover operation in a client / server system implementing virtual sockets according to one aspect is presented. This example assumes a failure is detected on the client device. During operation, the failover logic unit residing on the client device (e.g., ...) Figure 4 The failover logic unit 434 shown monitors various network interfaces to determine if the network interface has experienced a hardware failure (operation 602). If a failure is detected on the network interface, the failover logic unit can identify the network socket bound to the failed interface (operation 604). The network socket is encapsulated in a virtual socket (e.g., ...). Figure 4 Within the virtual socket 436 shown. The failover logic unit can notify the virtual socket that the identified network socket is bound to a faulty interface (operation 606). In response to the notification, the virtual socket can redistribute packets currently queued in the incoming queue of the identified socket to other incoming queues (operation 608). Depending on some aspects, these packets can be redistributed to other incoming queues by the load balancing logic unit. For example, they can be distributed based on the load of other incoming queues or in a round-robin manner. The identified network socket can then be removed from the virtual socket (operation 610). Therefore, packets sent to the write interface of the virtual socket will not be sent to the identified network socket. Figure 4 As an example, failover logic unit 434 can determine that network interface 412 has failed and notify virtual socket 436. In response, virtual socket 436 can redirect packets queued in incoming queue logic unit 420 to incoming queue logic unit 422. Virtual socket 436 can remove / deactivate network socket 416 and incoming queue logic 420, such that write interface 408 can forward all future incoming packets to incoming queue logic unit 422 serving network socket 418.

[0048] The failover logic unit can monitor the network interface and determine whether the failed network interface has returned to online (operation 612). If so, the network socket bound to the recovered interface can be reactivated and added to the virtual socket (operation 614). Adding the reactivated network socket back to the virtual socket may include providing memory for the incoming queue of the reactivated network socket. Alternatively, if the previously failed socket has been deleted (e.g., no reference to the recovered network interface is retained), a new network socket can be created to bind to the recovered interface.

[0049] from Figure 4 and Figure 5 As can be seen, the threads executing on both the client and server devices only know that the virtual socket is a single communication endpoint. These threads are unaware of the multiple network sockets encapsulated within the virtual socket. More specifically, the load balancing and failover operations occurring within the virtual socket are transparent to these threads. This arrangement can improve the communication throughput of the client / server system and make the connection fault-tolerant by ensuring continued communication in the event of hardware failure.

[0050] Figure 7 An exemplary computer system facilitating virtual sockets in a client / server system according to one aspect of this application is illustrated. The computer system 700 includes a processor 702, a memory 704, and a storage device 706. Furthermore, the computer system 700 may be coupled to a peripheral input / output (I / O) user device 710, such as a display device 712, a keyboard 714, and a pointing device 716. The storage device 706 may store an operating system 718, a virtual socket management system 720, and data 740.

[0051] The virtual socket management system 720 may include instructions that, when executed by the computer system 700, cause the computer system 700 or processor 702 to perform the methods and / or processes described in this disclosure. Specifically, the virtual socket management system 720 may include methods for encapsulating network sockets to create virtual sockets (e.g., ...). Figure 3A The instructions (socket encapsulation instructions 722) shown in the diagram for the virtual socket 306 are used to encapsulate the incoming and outgoing queues within the virtual socket (e.g., ...). Figure 3A The inbound queues 320-326 and outbound queues 328-338 shown in the diagram provide memory instructions (memory provision instructions 724), write interfaces for configuring virtual sockets, and read interfaces (e.g., ...). Figure 3A The write interfaces 308 and 312 shown in the figure are as follows Figure 3AThe instructions shown in the figure include: (interface configuration instruction 726) for reading interfaces 310 and 314; (load balancing instruction 728) for executing a load balancing strategy; (failover instruction 730) for performing a failover operation; (incoming packet forwarding instruction 732) for forwarding incoming packets to a socket-specific incoming queue based on a load balancing decision; (thread determination instruction 734) for determining the destination thread of a received packet; and (outgoing packet forwarding instruction 736) for forwarding outgoing packets to a thread-specific outgoing queue.

[0052] Generally, this disclosure provides a solution for load balancing and failover among multiple network sockets (e.g., TCP sockets) established between a client and a server. Virtual sockets can be created by encapsulating multiple network sockets. Virtual sockets can include write and read interfaces to allow client devices or servers to send and receive packets separately. Thread-safe multi-producer single-consumer queues can achieve lock-free synchronization between multiple threads executing on the client device and server. When an incoming packet is received at a virtual socket, the virtual socket's load balancing logic can select a network socket based on a predetermined load balancing strategy and forward the incoming packet to the corresponding socket-specific incoming queue. For outgoing packets, the virtual socket can determine its destination thread and forward the outgoing packet to the corresponding thread-specific outgoing queue. This approach allows for thread-safe packet exchange between the client and server while achieving load balancing and interface failover. The load balancing and failover operations are transparent to the processes executing on the client device and server. Aside from the storage system, this method can be applied to client / server systems of any size, provided that the number of threads on both the client and server can be predefined.

[0053] One aspect of this application provides a system and method for load balancing among multiple network sockets established between a local node and a remote node. During operation, the system can encapsulate multiple network sockets at the local node to form a local transport layer virtual socket including a write interface and a read interface. The system can receive packets at the write interface of the local transport layer virtual socket; select a network socket from the multiple network sockets based on a load balancing strategy; and forward the packets to a socket-specific inbound queue associated with the selected network socket to allow the packets to be sent via the selected network socket to the read interface of the corresponding remote transport layer virtual socket.

[0054] In a variation of this aspect, selecting a network socket may include determining the load of each network socket and selecting the network socket with the minimum load.

[0055] In another variation, determining the load of each network socket may include one of the following: determining the occupancy of the socket-specific inbound queue associated with each network socket, and determining the number of packets queued in the socket-specific inbound queue associated with each network socket.

[0056] In another variation, the selected network socket with the lowest load can correspond to a socket-specific incoming queue with the lowest occupancy or the smallest number of queued packets.

[0057] In a variation of this aspect, in response to detecting a failure of the network interface associated with a particular network socket, the system can remove the network socket from the local transport layer virtual socket and redistribute packets queued in the socket-specific inbound queue associated with the particular network socket to other socket-specific inbound queues associated with other network sockets encapsulated within the local transport layer virtual socket.

[0058] In another variation, in response to the detection of recovery of a failed network interface, the system can create a new network socket bound to the recovered network interface and encapsulate the new network socket into a local transport layer virtual socket.

[0059] In a variation of this aspect, the local node and the remote node can each execute multiple threads. The system can further receive packets associated with a specific thread executing on the remote node, determine the destination thread associated with the packets received from the remote node that is executing on the local node, and forward the packets received from the remote node to a thread-specific outgoing queue based on the destination thread.

[0060] In another variation, the system may further notify the destination thread to allow the destination thread to read packets received from the remote node from a thread-specific outgoing queue via a read interface.

[0061] In another variation, a thread-specific outgoing queue can be thread-safe and accessible by multiple network sockets.

[0062] In another variation, the socket-specific incoming queue can be thread-safe and accessible by multiple threads.

[0063] One aspect of this application provides an apparatus for load balancing among multiple network sockets established between a local node and a remote node. The apparatus may include a local transport layer virtual socket (LPV) encapsulating multiple network sockets. The LPV may include a write interface, a read interface, and multiple socket-specific inbound queues for storing packets received via the write interface. The apparatus may further include a load balancing logic unit for selecting a network socket from the multiple network sockets based on a load balancing strategy to transmit packets received via the write interface. The write interface of the LPV is used to forward the packets to a socket-specific inbound queue associated with the selected network socket to allow the packets to be sent via the selected network socket to the read interface of the corresponding remote LPV.

[0064] The methods and processes described in the Detailed Description section may be embodied in code and / or data, which may be stored in a computer-readable storage medium as described above. When a computer system reads and executes the code and / or data stored on the computer-readable storage medium, the computer system executes the methods and processes embodied in data structures and code and stored in the computer-readable storage medium.

[0065] Furthermore, the methods and processes described above may be included in hardware modules or devices. Hardware modules or devices may include, but are not limited to, application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGAs), dedicated or shared processors that execute specific software modules or code at specific times, and other programmable logic devices now known or developed later. When a hardware module or device is activated, it executes the methods and processes included therein.

[0066] The foregoing description has been presented for purposes of illustration and description only. It is not intended to be exhaustive or to limit the scope of this disclosure to its disclosed form. Accordingly, many modifications and variations will be apparent to those skilled in the art.

Claims

1. A computer-implemented method for load balancing between network sockets established between a local node and a remote node, the method comprising: A single write interface and a single read interface are presented to the local node using virtual sockets, wherein: The single write interface is configured to transmit packets to the remote node, and The single read interface is configured to receive the packet from the remote node; The virtual socket is used to encapsulate a set of Transmission Control Protocol (TCP) sockets between the local node and the remote node, wherein: The virtual socket includes a set of socket-specific incoming queues and a set of thread-specific outgoing queues. Each socket-specific incoming queue is coupled to a corresponding encapsulated TCP socket and is used to queue packets that will be transmitted through the corresponding encapsulated TCP socket. Each thread-specific outgoing queue is coupled to a set of encapsulated TCP sockets and is used to queue packets that will be received by the corresponding thread executing on the local node. Restrict access to the encapsulated set of TCP sockets to be accessed via the single write interface and the single read interface; Receive packets sent by the local node at the single write interface; Apply load balancing techniques to select encapsulated TCP sockets from a set of encapsulated TCP sockets; and The packet is forwarded to the socket-specific incoming queue corresponding to the selected encapsulated TCP socket.

2. The method as described in claim 1, wherein, Selecting the encapsulated TCP socket includes: Determine the payload for each encapsulated TCP socket; and Choose the encapsulated TCP socket with the lowest possible load.

3. The method as described in claim 2, wherein, The payload for each encapsulated TCP socket is determined to include one of the following: Determine the occupancy rate of the socket-specific incoming queue associated with each encapsulated TCP socket; and Determine the number of packets queued in the socket-specific incoming queue associated with each encapsulated TCP socket.

4. The method of claim 3, wherein, The selected encapsulated TCP socket with the minimum load corresponds to the socket-specific incoming queue with the lowest occupancy or the smallest number of queued packets.

5. The method of claim 1, further comprising: In response to detecting a failure of the network interface associated with a specific encapsulated TCP socket, the specific encapsulated TCP socket is removed from the virtual socket; as well as Packets queued in the socket-specific inbound queue associated with the specific TCP socket that was removed will be redistributed to other socket-specific inbound queues associated with other TCP sockets encapsulated within the virtual socket.

6. The method of claim 5, further comprising: In response to the detection of recovery of the failed network interface, a new TCP socket is created and bound to the recovered network interface; as well as The new TCP socket is encapsulated into the virtual TCP socket.

7. The method of claim 1, wherein, The local node and the remote node each execute multiple threads, and the method further includes: Receive packets associated with a specific thread executing on the remote node from the remote node; Determine the destination thread associated with the packet received from the remote node, which is executed on the local node; and The destination thread will forward the packets received from the remote node to a thread-specific outgoing queue.

8. The method of claim 7, further comprising notifying the destination thread to allow the destination thread to read the packet received from the remote node from the thread-specific outgoing queue via the read interface.

9. The method of claim 7, wherein, The thread-specific outgoing queue is thread-safe and can be accessed by the encapsulated set of TCP sockets.

10. The method of claim 7, wherein, The socket-specific incoming queue is thread-safe and can be accessed by the multiple threads.

11. The computer-implemented method of claim 1, further comprising: At the single read interface, a second packet sent by the remote node via an encapsulated TCP socket within an encapsulated set of TCP sockets is provided to the local node.

12. The computer-implemented method of claim 1, further comprising: Packets received from all threads on the client device will be forwarded to one of the socket-specific incoming queues in the set of socket-specific incoming queues.

13. An apparatus for load balancing between network sockets established between a local node and a remote node, the apparatus comprising: Local transport layer virtual socket, the local transport layer virtual socket is used for: The local node is presented with a single write interface and a single read interface, wherein: The single write interface is configured to transmit packets to the remote node. The single read interface is configured to receive the packet from the remote node; Encapsulate a set of TCP sockets between the local node and the remote node, wherein: The local transport layer virtual socket includes a set of socket-specific incoming queues and a set of thread-specific outgoing queues. Each socket-specific incoming queue is coupled to the corresponding encapsulated TCP socket and is used to queue packets that will be transmitted through the corresponding encapsulated TCP socket. Each thread-specific outgoing queue is coupled to a set of encapsulated TCP sockets and is used to queue packets that will be received by the corresponding thread executing on the local node. Restricting access to the encapsulated set of TCP sockets, accessible via the single write interface and the single read interface; and A non-transitory computer-readable medium storing instructions that, when executed by one or more processing resources, cause the one or more processing resources to apply load balancing techniques to select an encapsulated TCP socket from an encapsulated set of TCP sockets and forward the packet to a socket-specific incoming queue corresponding to the selected encapsulated TCP socket.

14. The apparatus of claim 13, wherein, Selecting the encapsulated TCP socket includes: Determine the payload for each encapsulated TCP socket; and Choose the encapsulated TCP socket with the lowest possible load.

15. The apparatus of claim 14, wherein, The instruction further causes the one or more processing resources to determine the payload of each encapsulated TCP socket in the following manner: Determine the occupancy rate of the socket-specific incoming queue associated with each encapsulated TCP socket; or Determine the number of packets queued in the socket-specific incoming queue associated with each encapsulated TCP socket.

16. The apparatus of claim 15, wherein, The selected encapsulated TCP socket with the minimum load corresponds to the socket-specific incoming queue with the lowest occupancy or the smallest number of queued packets.

17. The apparatus of claim 13, wherein, The instruction further causes the one or more processing resources to: In response to detecting a failure of the network interface associated with a specific encapsulated TCP socket, the specific encapsulated TCP socket is removed from the local transport layer virtual socket; and This causes packets queued in the socket-specific inbound queue associated with the specific TCP socket that was removed to be redistributed to other socket-specific inbound queues associated with other TCP sockets encapsulated within the local transport layer virtual socket.

18. The apparatus of claim 17, wherein, The instruction further causes the one or more processing resources to: In response to the detection of recovery of the failed network interface, a new TCP socket is created and bound to the recovered network interface; and The new TCP socket is encapsulated into the local transport layer virtual socket.

19. The apparatus of claim 13, wherein, The selected encapsulated TCP socket is used for: Receive packets associated with a specific thread executing on the remote node from the remote node; Determine the destination thread that is executed on the local node and associated with the packet received from the remote node; as well as The destination thread will forward the packets received from the remote node to a thread-specific outgoing queue.

20. The apparatus of claim 19, wherein, The selected encapsulated TCP socket is used for: The destination thread is notified to allow it to read the packets received from the remote node from the thread-specific outgoing queue via the read interface.

21. The apparatus of claim 19, wherein, The thread-specific outgoing queue is thread-safe and can be accessed by the encapsulated set of TCP sockets.

22. The apparatus of claim 19, wherein, The socket-specific incoming queue is thread-safe and can be accessed by multiple threads.

Images