Stateful services at the edge of a stateless cluster

Abstract

The present disclosure relates to stateful services on the edge of a stateless cluster. To enable dynamic scaling of network services at the edge, novel systems and methods are provided that enable the addition of new nodes or removal of existing nodes while maintaining the closeness of flows through the stateful services. The method provides a cluster of network nodes that can be dynamically resized to manipulate and process network traffic that utilizes stateful network services. Existing traffic flowing through the edge continues to work during and after cluster membership changes. All nodes in the cluster operate in active-active mode, i.e., they are receiving and processing traffic flows, thereby maximizing utilization of available processing power.

Description

[0001] This application is a divisional application of the invention patent application with application date of December 30, 2014, application number 201480083356.4, and invention name “Stateful Service on the Edge of a Stateless Cluster”. Technical Field

[0002] The present disclosure generally relates to stateful services on the edge of a stateless cluster. Background Art

[0003] Stateful network services such as firewalls and NAT are common at the edge of the network where traffic flows in and out. It provides basic security capabilities such as network security gateway services and web load balancing for performance and availability. As network traffic increases, the demand for edge services also increases, requiring additional resources to be allocated to edge services. Summary of the invention

[0004] In order to enable dynamic scaling of network services at the edge, some embodiments provide systems and methods that can add new nodes or remove existing nodes while maintaining the affinity of flows through stateful services. Some embodiments provide a network node cluster that can be dynamically resized to manipulate and process network traffic that utilizes stateful network services. Existing traffic flowing through the edge continues to work during and after cluster membership changes. In some embodiments, all nodes in the cluster operate in active-active mode, that is, they are receiving and processing traffic flows, thereby maximizing the utilization of available processing power.

[0005] In order to create a dynamically scalable node cluster that performs stateful services at the edge of a network (or network segment), some embodiments distribute flows to different nodes of the cluster so that all nodes of the cluster are actively participating in stateful services. In some embodiments, flows are hashed into the nodes of the cluster by a consistent hash function. The consistent hash function is on the nodes of the cluster (or based on the membership of the cluster) so that different flows will be distributed almost evenly to different nodes of the cluster. In some of these embodiments, each possible hash value generated by the consistent hash function is mapped to a node in the cluster, and each node is mapped to at least one possible hash value. Therefore, each node of the cluster is actively processing a stateful service for at least one flow. In order to ensure that stateful services are correctly performed for each flow, each flow is assigned to a node in the cluster during the life cycle of the flow. The node assigned to the flow manages and maintains state information related to the flow. In some embodiments, the identifier of the flow is used to guide the packets belonging to the flow to the assigned node.

[0006] After receiving a packet from the ECMP process, in some embodiments, the node identifies the owner node of the flow and forwards the packet to the identified owner node. In some embodiments, each node in the edge cluster performs consistent hashing on the flow identifier (e.g., a five-tuple of source (IP) address, source port, destination (IP) address, destination port, transport protocol) in the packet to identify the owner node of the flow.

[0007] In some embodiments, the owner node identified by the consistent hash of a particular flow is not the most appropriate edge node to perform stateful services on the flow. This is the case when the owner node of the flow identified by the consistent hash does not have the necessary state information required to correctly process the flow, but such state information is maintained at another node in the cluster. This other node can be the former owner node of the flow, or it can be the owner node of a related flow whose state information is needed to correctly process the flow. In order to provide consistent stateful services, in some embodiments, the owner node identified by the consistent hash delegates the packet to the node where the state information is maintained.

[0008] Some embodiments allow heterogeneous service support in a cluster. In some of these embodiments, the consistent hash function is based on the type of service that is called to service the flow. Specifically, in some embodiments, different sets of nodes are used for different services, that is, different services have different consistent hash functions based on different sets of nodes.

[0009] In some embodiments, flow delegation is based on a pinned flow table and an indirect table residing in a node of an edge cluster. Each entry in the indirect table and the pinned flow table corresponds to a flow. Each indirect table entry indicates that its edge node forwards or delegates a packet belonging to a flow according to the entry. Each entry in the pinned flow table for a flow indicates that its edge node locally processes the flow (therefore a fixed or anchored flow) and provides a stateful service for the flow, regardless of whether the node is the owner node of the flow. Rather than the owner node forwarding the packet to the delegation node where the state information of the flow is maintained, some embodiments forward the state information to the owner node.

[0010] In some embodiments, the identifier of the owner node is based not only on the hash function used, but also on the membership of the edge nodes in the cluster. In other words, determining which edge node is the owner node for a given flow is based in part on how many edge nodes are in the edge cluster that can provide stateful services. Therefore, any changes in membership in the edge cluster will change the result of the consistent hash. The size of the edge cluster can be changed to add new nodes or remove existing nodes. Some embodiments allow this to be done dynamically, i.e., while the edge cluster is actively processing network traffic that utilizes stateful network services, the size of the edge cluster is adjusted. Existing traffic flowing through the edge continues to be processed during and after the membership change of the cluster. In order to provide stateful services uninterruptedly by adjusting the size of the edge cluster, some embodiments anchor stateful services for existing flows at their original owner nodes until these existing flows are completed. In some embodiments, nodes in the edge cluster forward packets belonging to existing flows to their original owner nodes, and their original owner nodes still maintain the state information of the existing flows required for stateful services.

[0011] In some embodiments, the removal of edge nodes is done gracefully without interrupting stateful services provided by the edge cluster.For some of these embodiments, stateful processing of existing flows is maintained on their original owner nodes during the node removal process.

[0012] In some embodiments, delegation of flows is also necessary when state information of a first flow residing on a first node is necessary state information of a second flow assigned to a second node. In some embodiments, the first flow and the second flow are related flows assigned to different owner nodes via consistent hashing. In some embodiments, this occurs when the control flow and data flow of the same process are assigned to different owner nodes because they have different flow identifiers. To ensure that related flows operate within the same set of state information maintained on the same node, some embodiments use flow delegation to force related flows to be processed statefully by the same node, even if the two related flows are hashed to different owner nodes via consistent hashing.

[0013] In some embodiments, the owner node of the flow according to the original address pushes an indirect message to the owner node of the flow according to the converted DNAT address so as to keep the packets of the flow on the same node. As a result, the owner node identified by the converted DNAT address has an indirect table entry for delegating the packets of the flow to the owner node identified by the original address, which has a fixed flow table entry to receive and process packets with flow identifiers based on the DNAT address. In some other embodiments, instead of performing flow delegation, the edge cluster performing DNAT determines the owner node without using the destination address that can be converted. In other words, the cluster only applies consistent hashing to fields that are not affected by DNAT (e.g., source address field, source port field, protocol field).

[0014] In the case of SNAT, the edge nodes in the cluster change the source address of outgoing southbound packets. SNAT of the packets occurs after being routed through the owner node of its flow. In some embodiments, each node in the edge cluster uses its own unique SNAT address when performing address translation. Return northbound traffic is forwarded to the same owner node based on those unique SNAT addresses in the source address field. The owner node then applies reverse NAT while maintaining the state of the stateful SNAT process.

[0015] The foregoing summary is intended to serve as a brief introduction to some embodiments of the present invention. It is not meant to be an introduction or overview of all inventive subject matter disclosed in this document. The following detailed description and the drawings mentioned in the detailed description will further describe the embodiments described in the summary of the invention as well as other embodiments. Accordingly, in order to understand all the embodiments described in this document, a comprehensive review of the summary of the invention, the detailed description and the drawings is required. In addition, the subject matter claimed is not limited by the illustrative details in the summary of the invention, the detailed description and the drawings, but is to be defined by the appended claims, because the subject matter claimed may be embodied in other specific forms without departing from the spirit of the present subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The novel features of the invention are set forth in the appended claims. However, for purposes of illustration, several embodiments of the invention are set forth in the following figures.

[0017] Figure 1 A cluster of nodes in a network is conceptually illustrated, where network traffic is distributed to the nodes in the cluster for executing stateful services.

[0018] Figure 2a -b shows an edge cluster including a plurality of edge nodes for processing and / or filtering traffic in and out of the network.

[0019] Figure 3 Conceptually illustrates the distribution of a stateful service to a cluster of nodes based on consistent hashing of flows in the network.

[0020] Figure 4 Shown is forwarding packets between nodes in a cluster based on consistent hashing and ECMP.

[0021] Figure 5 Shows stateful processing of northbound and southbound traffic belonging to the same flow in the network.

[0022] Figure 6a -b conceptually illustrates a cluster in which flows are assigned to different sets of nodes based on the services provided.

[0023] Figure 7 A process for ensuring that a stateful process of a stream is consistently executed using the stream's owner node is conceptually illustrated.

[0024] Figure 8 Delegation of a packet from the owner node of a flow to another node within the edge cluster is shown.

[0025] Fig. 9 A process for determining whether an edge node should forward a received packet or perform a stateful process on the received packet is conceptually illustrated.

[0026] Fig.10 The forwarding of state information to the owner node identified by consistent hashing is shown.

[0027] Fig.11 It shows how changes in the membership of an edge cluster change the consistent hashing distribution of flows to edge nodes.

[0028] Fig.12a -b shows the forwarding of packets due to adding a new node to the edge cluster.

[0029] Fig.13 Conceptually illustrated is the ECMP process for an edge cluster during and after learning mode for a newly added node.

[0030] Fig.14 An example sequence of control messages between a network manager and an edge node is shown.

[0031] Fig.15 The process for managing these tables at the edge nodes during cluster resizing is conceptually illustrated.

[0032] Figure 16-17 A process for packet delegation between edge nodes in a cluster based on fixed flow tables, indirection tables, and new flow tables is conceptually illustrated.

[0033] Fig.18 An example node removal operation is conceptually illustrated in which an existing flow is anchored to its original owner node.

[0034] Fig.19 The operation of the edge cluster during and after a learning mode for removing a node is shown.

[0035] Fig. 20 The use of flow delegation to process related flows at the same node is shown.

[0036] Fig.21 A process for establishing flow delegation for statefully processing related flows in a cluster of edge nodes is conceptually illustrated.

[0037] Fig.22a An example DNAT service performed by an edge cluster is conceptually illustrated.

[0038] Figure 22b The use of flow delegation by an edge cluster to provide a stateful DNAT service is shown.

[0039] Fig.23a -b shows an edge cluster that does not use the destination address field for node allocation when performing DNAT.

[0040] Fig.24a -b shows how nodes in an edge cluster provide stateful SNAT services.

[0041] Fig.25 A data center is shown with host machines grouped into clusters or sections.

[0042] Fig.26 A data center is shown that includes a cluster or portion of host machines that primarily hosts edge service VMs that serve as edge nodes for compute VMs in other clusters.

[0043] Fig. 27 Logical networks for different tenants are shown, where each tenant is served by a set of edge service VMs that serve as an edge service cluster for the logical network.

[0044] Fig.28 An example host machine is shown operating virtualization software.

[0045] Fig.29 An edge service VM is shown running in a host machine.

[0046] Fig.30 An electronic system is conceptually illustrated that implements some embodiments of the present invention. DETAILED DESCRIPTION

[0047] In the following description, many details are set forth for the purpose of explanation. However, it will be appreciated by those skilled in the art that the present invention can be practiced without using these specific details. In other cases, well-known structures and devices are shown in the form of block diagrams in order to prevent the description of the present invention from being obscured by unnecessary details.

[0048] To enable dynamic scaling of network services at the edge, some embodiments provide systems and methods that can add new nodes or remove existing nodes while maintaining affinity for flows through stateful services. Some embodiments provide a cluster of network nodes that can be dynamically resized to manipulate and process network traffic that utilizes stateful network services. Existing traffic flowing through the edge continues to work during and after cluster membership changes. In some embodiments, all nodes in the cluster operate in active-active mode, i.e., they are receiving and processing traffic flows, thereby maximizing utilization of available processing power.

[0049] In order to create a dynamically scalable cluster of nodes that perform stateful services at the edge of a network (or network segment), some embodiments distribute flows to different nodes of the cluster so that all nodes of the cluster are actively participating in stateful services. In some embodiments, flows are hashed into the nodes of the cluster via a consistent hashing function. The consistent hashing function is on top of the nodes of the cluster (or based on the membership of the cluster) so that different flows will be distributed almost evenly to different nodes of the cluster. In some of these embodiments, each possible hash value generated by the consistent hashing function is mapped to a node in the cluster, and each node is mapped to at least one possible hash value. Therefore, each node of the cluster is actively processing a stateful service for at least one flow.

[0050] To ensure that stateful services are performed correctly for each flow, each flow is assigned to a node in the cluster during the lifecycle of the flow. The node assigned to the flow manages and maintains state information related to the flow. In some embodiments, the identifier of the flow is used to direct packets belonging to the flow to the assigned node. For some embodiments, Figure 1 A node cluster 110 in the network 100 is conceptually shown, where network traffic is distributed to nodes in the cluster for executing stateful services. Traffic distribution in the cluster supports affinities or correspondences between flows and their assigned nodes by using flow identifiers.

[0051] As shown, network 100 includes service cluster 110, which includes processing nodes 111-113 that can provide services to the network. In some embodiments, the services provided by cluster 110 include network address translation (NAT) and firewall. In some embodiments, these services are implemented at the edge of the network for processing and / or filtering the traffic in and out of the network. Therefore, these services are sometimes referred to as edge services, and the nodes that perform these edge services in cluster 110 are referred to as edge nodes. In some embodiments, the services provided by the nodes in cluster 110 are stateful services about streams. In other words, the nodes whose task is to provide the stateful services of streams need to maintain and manage the state of the streams.

[0052] A flow in some embodiments relates to a collection of packets belonging to a transport layer (i.e., layer 4 or L4) connection or session. In some embodiments, a node processing a flow includes a stateful engine to track protocol exchanges occurring on a transport layer connection in order to provide stateful services (e.g., making firewall assessments based on a set of rules). In some embodiments, the transport layer connection is based on the Transmission Control Protocol (TCP), and thus an L4 or transport layer connection is referred to as a TCP connection.

[0053] As shown, cluster 110 is receiving network traffic including packet 150, which includes headers for L2, L3, and L4 layers. In some embodiments, the L2 header of the packet includes source and destination MAC (media access control) addresses, the L3 header includes source and destination IP addresses, and the L4 header includes a source port, a destination port, and a transport protocol identifier. In some embodiments, the source IP address, the destination IP address, the source port, the destination port, and the protocol together form an identifier tuple (e.g., a five-tuple transport connection identifier) ​​for uniquely identifying a flow.

[0054] As mentioned, in some embodiments, each flow is assigned to a node so that the state information of the flow is maintained and managed by the node. In some embodiments, the node assigned to manipulate a specific flow is called the owner node of the flow. In order to ensure that the grouping belonging to the flow is processed by the same node (that is, the owner node of the flow) in the cluster, some embodiments use the flow identifier in the grouping to determine which node in the service cluster should be used for processing the grouping. As shown in the figure, flows "x1", "x2" and "x3" are assigned to 111 (node ​​A) based on the flow identifier of flows x1, x2 and x3 respectively. Flows "yl", "y2" and "y3" are assigned to 112 (node ​​B) based on the flow identifier of flows y1, y2 and y3 respectively. Based on flows z1, z2 and z3, the flow identifiers based on flows "z1", "z2" and "z3" are assigned to 113 (node ​​C). This allocation operation is conceptually shown by the node allocation module 160 based on flows in the cluster 110. In some embodiments, this allocation operation is performed by the computing device that realizes the node cluster or by each node 111-113 itself.

[0055] A node is a computing device or software running on a computing device that performs operations based on data in a network packet. Some network nodes receive and / or forward packets. Some network nodes perform calculations or other applications (e.g., computing nodes) based on data in a data packet. In some embodiments, a node is implemented by an independent physical router. In some embodiments, a node is a virtual machine implemented on a computing device that runs a hypervisor to serve as its host machine. In some embodiments, a computing device hosts multiple virtual machines as nodes in a network. Nodes 111-113 are edge service nodes that provide service operations (e.g., NAT or firewall) on packets between network 100 and the outside world (e.g., the Internet or other parts of the network).

[0056] Each of the nodes 111-113 can also provide the required stateful service for the packets belonging to the streams assigned to the nodes. For stateful services, in some embodiments, the node will create a state for each stream it processes. Therefore, the state of each stream is maintained by a node (that is, the owner node of the stream) in a consistent manner. As shown in the figure, node 111 (node ​​A) is the owner node of streams x1, x2 and x3 and maintains its state information for stateful processing. Similarly, node 112 (node ​​B) is the owner node of streams y1, y2 and y3 and maintains its state, and node 113 (node ​​C) is the owner node of streams z1, z2 and z3 and maintains its state. In some embodiments, the nodes of the same cluster (for example, the nodes 111-113 of cluster 110) belong to the same L2 segment, and packets can be forwarded to each other by L2 level protocols.

[0057] In some embodiments, the edge cluster 110 is defined by a container that includes a set of nodes. In some embodiments, in some embodiments, the node container (and therefore the corresponding edge cluster) is defined by the network manager using a VM template in some embodiments. The network administrator sends and receives events to the nodes in the container. The network manager also manages the life cycle of the nodes in the container. In some embodiments, a "north" container is defined for the cluster's interface facing "north" (with the computing nodes) to facilitate the configuration of resources (e.g., addresses of edge nodes) for southbound traffic. Similarly, a "south" container is defined for the cluster's interface facing "south" (with the outside world or the Internet) to facilitate the configuration of resources for northbound traffic.

[0058] Several more detailed embodiments of the present invention are described below. Section I further describes a system in which a flow-based stateful network service is distributed among edge service nodes in a cluster. Section II describes a method for dynamically adjusting the size of an edge service node cluster. Section III describes the processing of related flows by an edge service node cluster. Section IV describes address translation processing when a stateful service is distributed to an edge node cluster. Section V describes an edge service node implemented as a virtual machine hosted by a computing device. Section VI describes an example computing device used as a host machine for an edge node VM. Finally, Section IV describes an electronic system that implements some embodiments of the present invention.

[0059] I. Distribute stateful services to edge nodes

[0060] a.ECMP

[0061] As mentioned, cluster 110 is an edge cluster that provides edge services between network 100 and the outside world. In some embodiments, incoming traffic from the outside is distributed to any number of edge nodes using ECMP (equal cost multi-path routing). In some embodiments, edge nodes participate in ECMP with neighboring routers through the advertisement of BGP (Border Gateway Protocol) or OSPF (Open Shortest Path First) routes. Outgoing traffic entering the outside world can also be distributed to any edge node participating in the ECMP process.

[0062] Figure 2a -b shows an edge cluster 210, which includes multiple edge nodes 211-214 for processing and / or filtering traffic in and out of a network (or network segment) 200. A network (or network segment) 200 is a collection of interconnected computing and communication resources whose communication traffic with the outside world 290 (e.g., the Internet or other network segments) is controlled, monitored, or filtered by an edge cluster 210 logically located at the "edge" of the network 200 that interfaces with the outside world.

[0063] Some embodiments relate to network traffic entering the network from the outside world as southbound traffic and traffic leaving the network into the outside world as northbound traffic. As shown, the network includes a plurality of computing nodes 211-214 that receive network traffic from the outside world 290 through an edge cluster 210 and thus serve as a destination for northbound traffic. The computing nodes also generate traffic (or forwarding traffic) to the outside world 290 through the edge cluster 210 and thus are the origin of southbound traffic.

[0064] Figure 2a Northbound traffic entering network 200 from the outside world 290 (e.g., the Internet or other network segments) is shown entering compute nodes 221-224 through edge cluster 210. As shown, northbound traffic is distributed by ECMP router 240 to edge nodes 211-214 in edge cluster 210 for stateful processing. The processed northbound traffic is then forwarded to compute nodes 221-224 via L2 switching or L3 routing. Figure 2b Southbound traffic from compute nodes 221-224 through edge cluster 210 to the outside world 290 is shown. Southbound traffic is distributed by ECMP router 240 (or another ECMP router in network 200) to edge nodes 211-214 in edge cluster 210 for stateful processing. The processed southbound traffic is then forwarded to the outside world 290 (e.g., the Internet or other network segments).

[0065] b. Owner node

[0066] In some embodiments, traffic in both directions of a flow is assigned to be handled by the same node to ensure that stateful network services work correctly. Some embodiments assign packets to edge nodes based on flow identifiers. In some embodiments, ECMP is an operation independent of flow identification. There is no guarantee that ECMP will consistently distribute packets of the same flow to the same edge node for stateful processing. Therefore, some embodiments apply a stateless consistent hashing function to the flow identifier of each packet in order to identify the owner node of its flow. For each given flow, consistent hashing produces a consistent hash value for all packets of the flow. This ensures that packets belonging to the same flow are always processed statefully by the same node (i.e., the owner node of the flow), even if ECMP is used to distribute the packets.

[0067] Figure 3A cluster 310 is conceptually shown in which a stateful service is distributed to nodes based on consistent hashing of flows in a network 300. Cluster 300 includes nodes 311-314. As shown, packets 350 in network 300 are distributed to node 311 based on ECMP process 320. However, the flows to which these packets 350 belong are assigned to four nodes 311-314 according to consistent hashing function 330. As shown, flow F1 is hashed to node 311, flow F2 is hashed to node 312, flow F3 is hashed to node 313, and flow F4 is hashed to node 314. However, in some embodiments, it cannot be guaranteed that ECMP process 320 will route packets to the nodes to which the flows of the packets are hashed. For example, even if consistent hashing 330 will hash flow F2 to node 312, it cannot be guaranteed that the packets belonging to flow F2 are routed to node 312 by ECMP 320.

[0068] Upon receiving a packet from the ECMP process, in some embodiments, the node identifies the owner node of the flow and forwards the packet to the identified owner node. In some embodiments, each node in the edge cluster performs a consistent hash on the flow identifier in the packet (e.g., a five-tuple of source (IP) address, source port, destination (IP) address, destination port, transport protocol) to identify the owner node of the flow. Figure 4 It is shown that packets are forwarded according to consistent hashing of flows after ECMP routing.

[0069] Figure 4 The diagram shows forwarding packets between nodes 311-314 in cluster 310 based on consistent hashing and ECMP. Nodes 311-314 are the owner nodes of flows F1, F2, F3, and F4, respectively. ECMP distributes packets 451-454 to the nodes of cluster 310. The diagram shows forwarding of packets 451-454, respectively, in four phases 401-404.

[0070] In the first phase 401, the ECMP process routes packet 451 to node 311 (node ​​A). Packet 451 belongs to flow F2. Node 311 performs consistent hashing on the flow identifier of packet 451, and determines that the owner node of flow F2 is node 312 (node ​​B). Based on this determination, node 311 forwards packet 451 to node 312 for stateful processing, where the state of flow F2 is maintained. As shown in the figure, the state of flow F2 maintained at node 312 becomes "S1".

[0071] In the second phase 402, the ECMP process routes packet 452 to node 312 (node ​​B). Packet 451 also belongs to flow F2. Node 312 performs consistent hashing on the flow identifier of packet 452 and determines that the owner node of flow F2 is node 312 itself. Based on this determination, node 312 maintains packet 452 for stateful processing. As shown, due to the stateful processing of packet 452, the state of flow F2 at node 312 has been updated to "S2".

[0072] In the third stage 403, the ECMP process routes packet 453 to node 313 (node ​​C). Packet 453 belongs to flow F4. Node 313 performs consistent hashing on the flow identifier of packet 453 and determines that the owner node of flow F4 is node 314 (node ​​D). Based on this determination, node 313 forwards packet 453 to node 314 for stateful processing, wherein the state of flow F4 is maintained. As shown in the figure, the state of flow F4 maintained at node 312 becomes "S1". On the other hand, the state of flow F2 remains "S2" at node 312.

[0073] In the fourth stage 404, the ECMP process routes packet 454 to node 314 (node ​​D). Packet 454 belongs to flow F2. Node 314 performs consistent hashing on the flow identifier of packet 454 and determines that the owner node of flow F2 is node 312. Based on this determination, node 314 forwards packet 454 to node 432 for stateful processing, where the state of F2 is maintained. As shown in the figure, the state of flow F2 has changed to "S3".

[0074] As mentioned, in some embodiments, the edge cluster handles northbound traffic entering the network and southbound traffic leaving the network. Regardless of the direction of the traffic and the ECMP process employed, consistent hashing is used to consistently identify the owner node of the flow. In other words, regardless of the direction of the traffic, packets belonging to the same flow will be processed statefully at the same owner node.

[0075] Figure 5 Stateful processing of northbound and southbound traffic belonging to the same flow in network 300 is shown. Figure 5 Shown is a forwarding path for northbound packet 551 and a forwarding path for southbound packet 552. Both packets 551 and 552 belong to flow F2, whose owner node is node B 312.

[0076] As shown, northbound packet 551 is routed by ECMP to node C 313. Node C in turn forwards northbound packet 551 to node B 312 through consistent hashing. Southbound packet 552 is routed by ECMP to node A 311. Node A 311 then forwards packet 552 to node B 312 through consistent hashing. By using consistent hashing, both node C and node A recognize the same owner node (node ​​B 312) of flow F2 even though packets 551 and 552 are in different directions.

[0077] Some embodiments allow heterogeneous service support in a cluster. In some of these embodiments, the consistent hash function is based on the type of service that is called to serve the flow. Specifically, in some embodiments, different sets of nodes are used for different services, that is, different services have different consistent hash functions based on different sets of nodes. For example, some embodiments have a first set of nodes for ALG, a second set of nodes for NAT, and a third set of nodes for firewalls. Therefore, when a packet arrives, the node maps the flow of the packet to a service, such as NAT, and uses the input node set (supplied for NAT) for consistent hashing, taking care of the NAT service, and forwarding the packet to the owner node according to the hash result. In some embodiments, different services are assigned different sets of nodes in the cluster based on, for example, whether the service is a "slow path" service (e.g., ALG) or a "fast path" service (e.g., firewall).

[0078] Figure 6a -b conceptually illustrates a cluster 600 in which flows are assigned to different sets of nodes based on the services provided. Specifically, the owner node of a flow of incoming packets is identified by a consistent hash specific to the service provided. Figure 6a As shown, a first consistent hash function 621 is used to distribute flows to nodes in cluster 600 for a first edge service (eg, ALG), while a second consistent hash function 622 is used to distribute flows to nodes in cluster 600 for a second edge service (eg, firewall).

[0079] Cluster 600 has nodes 611-617 (nodes A to G). As shown, a first consistent hash function 621 is a hash function on nodes 611, 612, 614, and 615 (nodes A, B, C, E). A second consistent function 622 is a hash function on nodes 612, 613, 615, 616, 617 (nodes B, C, E, F, G). In other words, flows using the first edge service are only assigned to nodes A, B, C, and E, and flows using the second edge service are only assigned to nodes B, C, E, F, and G. Therefore, as Figure 6bAs shown, nodes A, B, D and E are maintaining the state of the flow using the first edge service, while nodes B, C, E, F and G are maintaining the state of the flow using the second edge service. In some embodiments, some nodes in the cluster can provide a variety of different edge services. For example, nodes 612 and 615 (B and E) provide stateful edge services to both the first edge service and the second edge service, and can therefore be selected by the first consistent hash function 621 or the second consistent hash function 622. In some embodiments not shown, each node in the cluster can only provide one type of edge service.

[0080] For some embodiments, Figure 7 Conceptually illustrated is a process 700 for ensuring that a stateful process for a flow is consistently executed using the owner node of the flow, regardless of where the packets of the flow are initially routed (e.g., by ECMP). In some embodiments, the process 700 is performed by an edge node in an edge service cluster. The process 700 begins when it receives a packet at the edge of the network, either southbound out to the outside world or northbound into the network. In some embodiments, the packet is initially distributed to the edge node by ECMP.

[0081] The process identifies (at 710) the flow of the incoming packet. In some embodiments, this identification is based on a flow identifier in the packet, such as source / destination address, source / destination port, and protocol fields in the L3 and L4 headers of the packet.

[0082] The process then computes (at 720) a consistent hash for the identified flow by, for example, hashing the flow identifier into a consistent hash value. In some embodiments, different consistent hash functions are used for different services, as described above with reference to FIG6. In some of these embodiments, the process uses the flow identifier of the incoming packet to identify a particular edge service, and then applies a corresponding consistent hash function to the identified edge service.

[0083] The process then maps (at 730) the consistent hash value to identify the owner node of the flow. Next, the process determines (at 740) whether the edge node running process 700 is the owner node of the flow. If the edge node itself is the owner node of the flow, the process proceeds to 750 to locally perform stateful services on the packet based on the flow. If the edge node itself is not the owner node of the flow, the process proceeds to 760 to forward the packet to the identified owner node so that the owner node can perform stateful services based on the flow. The process 700 then ends.

[0084] c. Delegation Flow

[0085] In some embodiments, the owner node identified by the consistent hash of a particular flow is not the most appropriate edge node to perform stateful services on the flow. This is the case when the owner node of the flow identified by the consistent hash does not have the necessary state information required to correctly process the flow, but such state information is maintained at another node in the cluster. This other node can be the former owner node of the flow, or it can be the owner node of a related flow whose state information is needed to correctly process the flow. In order to provide consistent stateful services, in some embodiments, the owner node identified by the consistent hash delegates the packet to the node where the state information is maintained.

[0086] In some embodiments, flow delegation is based on fixed flow tables and indirection tables residing in nodes of edge clusters. Each entry in the indirection table and the fixed flow table corresponds to a flow. Each indirection table entry indicates that its edge node forwards or delegates the packet belonging to the flow according to the entry. Each entry in the fixed flow table for a flow indicates that its edge node locally handles the flow (therefore fixing or anchoring the flow) and provides a stateful service for the flow, regardless of whether the node is the owner node of the flow.

[0087] Figure 8 The delegation of a packet from the owner node of a flow to another node within an edge cluster 810 is shown. The edge cluster includes edge nodes 811-814 (nodes A, B, C, and D, respectively), each of which can provide stateful services for flows at the edge. Edge node 812 (node ​​B) is assigned as the owner node of flow "F2" through consistent hashing. However, owner node B 812 has delegated flow F2 to node D 814 as the delegated node, so that node D, rather than node B, will handle the stateful processing of flow F2.

[0088] As shown, the delegation of flow F2 is completed according to the indirect table 820 and the fixed flow table 830 implemented in the owner node B (owner node of flow F2) and node D (delegated node), respectively. The indirect table 820 of node B has an entry 821 for flow F2 pointing to node D. The entry 821 indicates that node B forwards the packet belonging to flow F2 to node D. The fixed flow table 830 of node D has an entry 831 indicating that node D performs stateful services on the packet belonging to flow F2, even if node D is not the owner node of F2. Node D (814) also has a local flow table 840 that tracks the state of the flow (including fixed flow F2) it is processing. In some embodiments (not shown), the local flow table and the fixed flow table are implemented as one table so that the edge node can use the entries in the local flow table to determine which flows are fixed to the node.

[0089] Figure 8Also shown are example packet forwarding and flow delegation operations in edge cluster 810. The example shows the forwarding of packet 851 belonging to flow "F2".

[0090] As shown, the ECMP process 851 forwards packet 851 to node C (813). However, node C is neither the owner node of flow F2 nor a node with F2 as a fixed flow. Since node B is the owner node of F2 according to consistent hashing, packet 851 is forwarded to node B. However, node B's indirect table 820 has an entry 821 for flow F2 pointing to node D, so node B forwards packet 851 to node D. Although node D is not the owner node of F2 according to consistent hashing, it still accepts and performs stateful service for packet 851. This is because the fixed flow table 830 of node D has an entry 831 for flow F2. (Although not shown, if the ECMP process directly routes the F2 packet to node D, node D will immediately perform stateful service based on its fixed flow table entry for F2 instead of forwarding the packet based on consistent hashing.)

[0091] For some embodiments, Fig. 9 Conceptually illustrated is a process 900 for determining whether an edge node should forward a received packet or perform a stateful process on a received packet. Process 900 begins when an edge node receives an incoming packet. Such a packet can be part of northbound traffic entering the network or southbound traffic leaving to the outside world (e.g., the Internet, other parts of a data center, or other segments of a network).

[0092] The process identifies (at 910) the flow to which the incoming packet belongs. In some embodiments, the flow of the packet can be identified from the address / port field in the L2-L4 header of the packet (e.g., a five-tuple). The process then determines (at 915) whether the identified flow is in the fixed flow table of the edge node. The corresponding entry in the fixed flow table indicates to the edge node that the flow is to be processed statefully locally, regardless of the consistent hashing. If the identified flow is in the fixed flow table (i.e., has a corresponding entry), the process proceeds to 950 to perform stateful services on the incoming packet. Otherwise, the process proceeds to 920.

[0093] At 920, the process performs consistent hashing and calculates a hash value for the flow. The calculated hash value is then used to determine or identify the owner node of the flow. Consistent hashing is limited to nodes in the edge cluster, so each flow will map to only one node in the cluster.

[0094] Then, the process determines (at 930) whether the edge node executing the process itself is the owner node of the identified flow. If so, the process proceeds to 940. If the edge node is not the owner of the identified flow, the process proceeds to 970.

[0095] At 940, the process determines whether the identified flow is in the indirect table of the edge node. In this operation, the process has determined that the edge node running the process is the owner node of the flow identified by consistent hashing. However, the corresponding entry in the indirect table will indicate to the edge node that the flow must be forwarded elsewhere for stateful processing. If the identified flow has a corresponding entry in the indirect table, the process proceeds to 960. Otherwise, the process proceeds to 950 to perform stateful services on the incoming packets as the owner node of the identified flow.

[0096] At 950, the process performs the required stateful services (e.g., firewall, NAT) on the packet. In some embodiments, this operation is based on state information maintained by the edge node. Then, based on the content or type of the packet (e.g., SYN, ACK packets at the beginning of a TCP connection), the maintained state information is updated accordingly. Then the process 900 ends.

[0097] At 960, the process forwards the packet to a delegated node in the cluster for stateful processing (even if the edge node running the process is the owner node of the flow). In some embodiments, the delegated node is identified by an entry in the indirection table for the identified flow (e.g., entry 821 in the indirection table 820 identifying node D for F2). The process 900 then ends.

[0098] At 970, the process forwards the packet to the owner node of the flow. In some embodiments, the nodes of the cluster are in the same L2 segment, so that the forwarding is based on the L2 protocol. The process 900 then ends.

[0099] d. Transmit status information

[0100] As described above, in some embodiments, a node that is determined to be the owner node of a flow by consistent hashing does not have the necessary state information to correctly process the flow. Section 1c above describes a method in which an owner node delegates packets belonging to a flow to a node that is maintaining the necessary state information (e.g., a previous owner node). Instead of the owner node forwarding packets to a delegation node that maintains the state information of the flow, some embodiments forward the state information to the owner node.

[0101] For some embodiments, Fig.10The forwarding of state information to the owner node identified by consistent hashing is shown.The figure shows the forwarding of state information in the edge cluster 810 in three stages 1001-1003.

[0102] The first stage 1001 shows the state information of flow F4 maintained at its original owner node 814 (node ​​D). Because node 814 has performed a stateful service for flow F4, it has state information for the flow.

[0103] The second stage 1002 shows that a different node, namely Node B (812), is selected as the owner node of flow F4 through consistent hashing. This change in owner node assignment can occur due to changes in the membership of the edge cluster (e.g., node addition or removal) or due to owner node assignment overrides (e.g., related flows). However, the state information for F4 is still maintained at Node D, so Node B cannot correctly provide stateful services to flow F4 without the state information.

[0104] The third stage 1003 shows the forwarding of state information for flow F4. In some embodiments, this forwarding is through control plane messages between nodes in the cluster, or through control messages using a network manager. The state information of F4 is transmitted (or copied) from node D to node B, so that the information will be available to node B to provide stateful services for flow F4.

[0105] II. Dynamically resize the edge node cluster

[0106] In some embodiments, the identification of the owner node is based not only on the hash function used, but also on the membership of the edge nodes in the cluster. In other words, determining which edge node is the owner node for a given flow is based in part on how many edge nodes in the edge cluster are capable of providing stateful services. Therefore, any changes in membership in the edge cluster will change the results of the consistent hashing. For example, when there are M nodes in the edge cluster, the consistent hashing will identify the first node as the owner node of the flow, but when there are N nodes in the cluster, a different second node will be identified as the owner node of the flow.

[0107] As described above, the size of an edge cluster can be changed to add new nodes or remove existing nodes. Some embodiments allow for this dynamic adjustment of the size of an edge cluster, i.e., while the edge cluster is actively processing network traffic that utilizes stateful network services. Existing traffic flowing through the edge continues to be processed during and after the membership change of the cluster. In order to provide stateful services uninterruptedly by adjusting the size of the edge cluster, some embodiments anchor stateful services for existing flows at their original owner nodes until these existing flows are completed. In some embodiments, nodes in the edge cluster forward packets belonging to existing flows to their original owner nodes, which still maintain the state information of the existing flows required for the stateful services.

[0108] a. Add edge nodes

[0109] Fig.11 1101-1102 shows how changes in the membership of the edge cluster 810 change the consistent hashing distribution of flows to edge nodes. Specifically, the figure shows the addition of a new fifth node E (815) to the edge cluster in two stages 1101-1102. Figure 8 The edge cluster 810 initially has four nodes 811-814 (nodes A, B, C, D).

[0110] The first stage 1101 shows the allocation of owner nodes by consistent hashing before adding node E. As shown, before adding node E, the consistent hashing of the flow is on nodes A, B, C and D to identify the owner node from among these four nodes. Under this consistent hashing function, flows F1 and F5 are assigned to node A (811), flows F2 and F6 are assigned to node B (812), flows F3 and F7 are assigned to node C (813), and flows F4 and F8 are assigned to node D (814). For this example, it is assumed that flows F1-F8 are active, ongoing flows that are statefully processed by nodes A, B, C and D.

[0111] The second stage 1101 shows the addition of a new node E and its impact on consistent hashing. With the addition of node E, consistent hashing is performed on nodes A, B, C, D and E to identify the owner node from among these five nodes. Therefore, some of the streams F1-F8 are reallocated to different owner nodes. In this example, each stream except F8 is assigned different owner nodes by consistent hashing. Node A is now the owner node of F3, node B is now the owner node of F1, node C is now the owner node of F4 and F6, and node D is now the owner node of F7 and F8. The newly added node is assigned to the owner node of F2 and F5.

[0112] Although the addition of the new node has caused the existing flows (F1-F8) to be assigned to the new owner node, the state information of the existing flows is still maintained at their original owner node. In order to correctly continue stateful service for these flows, some embodiments use indirection tables and fixed flow tables to delegate packets, so packets belonging to existing flows are processed with the correct state information.

[0113] Fig.12a -b shows the forwarding of packets due to the addition of a new node 815 to the edge cluster 810. This addition occurs while node B (812) is still performing stateful services for flow F2, which was the original owner node of flow F2 before the new node was added. The figure shows the processing of node addition in four stages 1201-1204.

[0114] The first stage 1201 shows the state of the edge cluster 810 before adding node E (815). As shown, node B 812 is the owner node of flow F2 through consistent hashing. It is maintaining state information related to F2, and even when ECMP initially routes the packet elsewhere, the packet belonging to F2 is forwarded to node B for stateful processing.

[0115] The second stage 1202 shows the processing of flow F2 by the cluster 810 immediately after the addition of a new node (node ​​E 815). The addition of node 815 (node ​​E) causes the consistent hashing to identify the new node 815 as the owner of flow F2, rather than node B 812 (the original owner node of F2). However, the state information for F2 is still maintained at node B. Node E 815 (the new owner node of F2 by consistent hashing) therefore forwards packets in flow F2 to node B 812 (the original owner node of F2) for stateful service, rather than performing the stateful service itself.

[0116] As shown, node E has an indirect table 1220 with an entry 1221 for F2 pointing to node B. Any packets that arrive at node E belonging to F2 (whether through ECMP or consistent hashing) will be forwarded / delegated to node B according to this indirect table entry. In addition, node B has a fixed flow table 1230 with an entry 1231 for flow F2. This entry instructs node B to keep any packets belonging to F2 for processing rather than forwarding them elsewhere, even though it is no longer the owner node through consistent hashing.

[0117] The third stage 1203 shows the processing of the new flow F9 during the addition of node E. A new flow (e.g., F9) is a flow that starts after the cluster is resized. Such a flow does not have an original owner node, and it does not have the existing state information required for correct stateful processing. Therefore, the new flow can be processed by its owner node according to consistent hashing without delegation. In some embodiments, each node maintains a new flow table (e.g., 1240) to track the new flow added during the cluster resizing. This is necessary because in some embodiments, the newly added node is not recognized by the ECMP process and cannot receive packets directly from the ECMP. The node receiving the packet from the ECMP can refer to the new flow table and forward the packet directly to the owner node, even if the owner node is a newly added node. In this example, node C (813) receives packet 1253 from ECMP and forwards the packet directly to node E based on the new flow table 1240 of node C, where the new flow table 1240 has an entry 1241 corresponding to the new flow F9. Although Node E is a new node in the cluster, it is able to process flow F9 immediately as the owner node of F9 because F9 is a new flow that can be processed without considering existing state information.

[0118] The fourth stage 1204 shows the completion of flow F2 by node B. Since F2 is an existing flow before adding node E, its completion also makes it unnecessary for node E to delegate the F2 packet to node B based on its indirect table 1220. Node B also does not have to maintain entries in its fixed flow table 1230 for F2. Therefore, the addition of the new node is considered complete. However, the new flow F9 will continue to be processed by node E until it is completed. However, in some embodiments, when the owner node of F9 (node ​​E) joins ECMP and is therefore no longer considered a new node, the new flow table in the cluster (e.g., the new flow table 1240 in node C) will clear its entries related to F9.

[0119] After adding a new node, the edge cluster is considered to be in learning mode for the new node until the new node has joined the ECMP process and starts receiving packets directly from ECMP. Once this occurs, new flow tables are no longer needed because nodes in the cluster (including the newly added node) can receive ECMP packets and can forward packets to the legitimate owner of its flow based on consistent hashing across all nodes in the cluster. Fixed flow tables and indirection tables can remain operational until those pre-existing flows (flows before the node was added) complete their operations. Fig.13 The ECMP process for an edge cluster 810 is conceptually illustrated during and after a learning mode for a newly added node. The figure shows the addition of a node E in four stages 1301-1304.

[0120] Before the cluster is resized (i.e., steady state, not shown in the figure), node A is providing stateful service for flows F1 and F5, and node B is providing stateful service for flow F2, because nodes A and B are the owner nodes of F1, F5, and F2 according to consistent hashing on nodes A, B, C, and D. The ECMP process is distributing packets to all four nodes in the cluster.

[0121] The first stage 1301 shows the edge nodes immediately after adding node E, i.e., the cluster 810 has just entered the learning mode of node E. Under the new consistent hashing on nodes A, B, C, D, and E, the new owner of flow F1 is node B, and the new owner of flows F2 and F5 is node E. As existing flows started before adding node E, flows F1, F2, and F5 are anchored to their original owner nodes through the fixed flow table and the indirect table. The ECMP process simultaneously continues to distribute packets to only nodes A, B, C, and D. The new node E will not receive packets directly from the ECMP process during the learning mode.

[0122] The second stage 1302 shows that flow F5 is completed by node A. This means that node A no longer needs to process F5 as a fixed flow, and node E no longer needs to delegate packets of F5 to node A according to its indirection table.

[0123] The third stage 1303 shows the addition of a new flow F9, whose owner node is the newly added node E. As shown, the ECMP process at this stage still does not distribute packets to the newly added node E, so other nodes in the cluster (nodes A, B, C, D) will have to forward the ECMP packets of F9 to node E for stateful processing based on the entry for F9 in the new flow table.

[0124] The fourth stage 1304 shows the cluster 810 in the new stable state after the learning mode of node E has been completed. The ECMP process now includes node E, so that incoming packets can be distributed to all nodes in the cluster, including the newly added node E. The new flow F9 also continues to be processed by node E, but the entry for F9 in the new flow table in cluster 810 is no longer needed and is therefore cleared. Fixed flows such as F1 and F2 (i.e., flows that pre-existed before adding node E) continue to be statefully served in their original owner nodes through the indirect table and the fixed flow table until those flows are completed.

[0125] In some embodiments, adding new nodes to the cluster is controlled by a network manager. In some embodiments, the network manager controls the resizing of the edge cluster by exchanging control plane messages with the nodes in the edge cluster. Fig.14An example sequence of control messages 1401-1414 between a network manager 1499 and edge nodes 811-815 (nodes A, B, C, D, and E) is shown for adding a new node 815 (node ​​E) to an edge cluster 810. Prior to the message exchange sequence, only nodes 811-814 (nodes A, B, C, and D) are active in cluster 810.

[0126] Manager 1499 sends messages 1401-1404 to nodes 811-814, respectively, notifying these nodes that a new node (node ​​E) is to be added to the cluster. Manager 1499 also sends message 1405 to the newly added node 815 (node ​​E). It is used to inform the new node (node ​​E) which nodes are in the cluster, which will include nodes A, B, C, D and E.

[0127] Next, node A sends a message 1406 to node E, instructing node E to forward packets in flow F5 to node A (e.g., by creating an entry in its indirection table). On the other hand, node B sends a message 1407 to node E, instructing node E to forward packets in flow F2 to node B. Messages 1406 and 1407 are generated because the addition of the new node E has caused the owner node assignments of flows F5 and F2 to be transferred from nodes A and B to node E, as described above with reference to Figure 11-13 Nodes A and B then continue to process F5 and F2 as fixed flows.

[0128] When node A has completed the fixed flow F5, it sends a message 1408 to the manager 1499 to indicate that it has completed the fixed flow F5. Similarly, when node B has completed the fixed flow F2, it sends a message 1409 to the manager 1499 to indicate that it has completed the fixed flow F2. Since flows F2 and F5 are fixed flows whose owner nodes are the newly added nodes (node ​​E), the manager will understand that the learning mode of node E is complete and node E can participate in ECMP. Therefore, the manager 1499 sends a message 1410 to node E, notifying it to start the ECMP process.

[0129] In this example (i.e., Figure 11-13 ), new flow F9 is hashed to a new node (node ​​E) during learning mode, and thus each existing node has a new flow table entry for flow F9 when the new flow is hashed to the new node. After learning mode for node E is completed, manager 1499 sends messages 1411-1414 to nodes 811-814, respectively, to clear any entries for flow F9 in their new flow tables.

[0130] As mentioned, by adjusting the size of the edge cluster, the continuous, uninterrupted stateful service of active flows relies on the delegation of packets between the nodes selected as the owner nodes of those flows and the nodes that still maintain the state information of those flows. In some embodiments, these packet forwarding operations rely on a set of tables, such as the indirection table, the fixed flow table, and the new flow table, as described above with reference to Figure 8 and 12. For some embodiments, Fig.15 A process 1500 for managing these tables at an edge node during cluster resizing is conceptually illustrated.

[0131] Process 1500 begins when process 1500 detects (at 1510) a change in membership in an edge cluster. This change in membership results in a change in flow-to-node assignment via consistent hashing. In some embodiments, the edge node knows about such a change because it has received control plane messages (e.g., messages 1401-1405) from the network controller that a new node has been added to the cluster.

[0132] The process then identifies (at 1520) active flows that are incomplete (i.e., still active) in the edge node but are assigned to other nodes due to consistent hashing changes. Next, the process adds (at 1530) the identified flow as an entry to its fixed flow table. This notifies the edge node to perform stateful services on packets of the identified flow, even though it is no longer the owner node of the flow. The process then sends an indirect message (at 1540) to the owner node of the identified flow so that the owner node will know to delegate the flow to this edge node.

[0133] Next, the process identifies (at 1550) active flows that are incomplete (i.e., still active in other nodes) but hashed to the node. In some embodiments, this is based on received indirect messages (e.g., Fig.14 The indirect messages are generated by edge nodes that have incomplete flows that have been reassigned to the edge node. The process then adds (at 1560) these identified flows as entries to the indirect table. The edge node running the process will have to delegate the grouping of these flows according to those entries in the indirect table.

[0134] With the fixed flow table and the indirection table updated, the process then forwards (at 1570) the packet or performs stateful services on the packet by referencing the entries in the indirection table, the fixed flow table, and the new flow table. In some embodiments, when the edge node encounters a newly started flow, the entry in the new flow table is updated. The management of the new flow table will be described below with reference to Fig.16Further described. The process then reports (at 1580) the completed fixed flows, and removes (at 1590) the corresponding entries for those completed flows in the fixed flow table and the indirect table. In some embodiments, the edge node that completes the fixed flow reports the flow completion via a control plane message, so the edge node with the corresponding entry in its indirect table will know to remove those entries and stop delegating those flows. Some embodiments include a timeout mechanism that automatically removes entries in the indirect table and the fixed flow table after the corresponding flow is inactive for a period of time. Then process 1500 ends.

[0135] Figure 16-17 Processes 1600 and 1700 are conceptually illustrated for packet delegation between edge nodes in a cluster based on fixed flow tables, indirection tables, and new flow tables. Fig.16 The process 1600 is performed by an existing edge node during a learning mode of a newly added node, while Fig.17 The process 1700 is performed by the newly added node during the corresponding learning mode.

[0136] Fig.16 The process 1600 begins by identifying (at 1610) a flow of an incoming packet. In some embodiments, the flow of a packet may be identified from the address / port field in the L2-L4 header of the packet (e.g., a five-tuple). The process then determines (at 1615) whether the identified flow is in the fixed flow table of the edge node. The corresponding entry in the fixed flow table indicates to the edge node that the flow is to be processed statefully locally, regardless of consistent hashing. If the identified flow is in the fixed flow table (i.e., has a corresponding entry), the process proceeds to 1650 to perform stateful services on the incoming packet. Otherwise, the process proceeds to 1620.

[0137] At 1620, the process performs consistent hashing and generates a hash value for the flow. The calculated hash value is then used to determine or identify the owner node of the flow. The consistent hashing includes the newly added node, so the hash result can include the newly added node.

[0138] The process then determines (at 1630) whether the edge node executing the process itself is the owner node of the identified flow. If so, the process proceeds to 1640. If the edge node is not the owner of the identified flow, the process proceeds to 1670.

[0139] At 1640, the process determines whether the identified flow is in the indirect table of the edge node. In this operation, the process has determined that the edge node running the process is the owner node of the flow identified by consistent hashing. However, the corresponding entry in the indirect table will indicate to the edge node that the flow must be forwarded elsewhere for stateful processing. If the identified flow has a corresponding entry in the indirect table, the process proceeds to 1660. Otherwise, the process proceeds to 1650 to perform stateful services on the incoming packets as the owner node of the identified flow.

[0140] At 1650, the process performs the required stateful services (e.g., firewall, NAT) on the packet. In some embodiments, this operation is based on state information maintained by the edge node. The maintained state information is then updated accordingly based on the content of the packet. The process 1600 then ends.

[0141] At 1660, the process forwards the packet to a delegated node in the cluster for stateful processing (even if the edge node running the process is the owner node of the flow). In some embodiments, the delegated node is identified by an entry in the indirect table for the identified flow. The process 1600 then ends.

[0142] At 1670, the process determines whether the owner node based on consistent hashing is a newly added node. If the owner node is a newly added node, the process proceeds to 1680. Otherwise, the process 1600 proceeds to 1675 to forward the packet to the owner node of the flow.

[0143] At 1680, the process determines whether the flow is a new flow assigned to the new node. Unlike existing flows that have existing state information stored in their original owner nodes, new flows have no original owner nodes or pre-existing state information, and can therefore begin stateful processing at their owner nodes at will, whether a newly added node or an existing node. In some embodiments, the process determines whether the flow is a new flow by checking the entries in its new flow table. If the packetized flow is a newly added flow, the process proceeds to 1685. If the flow is not a newly added flow (i.e., a pre-existing flow), the process proceeds to 1690.

[0144] At 1685, the process updates the new flow table at the edge node, i.e., if there is no entry for the flow in the new flow table, the process adds the corresponding entry to the new flow table. The process then proceeds to 1675 to forward the packet to the owner node of the new flow.

[0145] At 1675, the process forwards the packet to the owner node of the flow, which is the node identified based on the consistent hash including the newly added node. The process 1600 then ends.

[0146] At 1690, the process computes a hash of the flow only on existing nodes (i.e., by excluding the newly added nodes). The hash yields the original owner node of the flow. This operation is required if the flow is an existing flow and the edge node does not yet have a corresponding entry in its indirect table. Therefore, the process must compute a hash value based on the existing nodes in the cluster (excluding the newly added nodes) in order to determine / reconstruct which node is the original owner of the flow. The process then forwards the packet to this original owner based on the computed hash. The process 1600 then ends.

[0147] Fig.17 The process 1700 begins by identifying (at 1710) a flow of incoming packets. In some embodiments, the flow of packets can be identified from address / port fields in the L2-L4 header of the packet (eg, a five-tuple).

[0148] At 1720, the process performs consistent hashing and calculates a hash value for the stream. The calculated hash value is then used to determine or identify the owner node of the stream. The consistent hash includes the newly added node, so the hash result can include the newly added node. The process then determines (at 1730) whether the newly added node itself is the owner node of the identified stream. If so, the process proceeds to 1740. If the newly added node is not the owner of the identified stream, process 1700 ends because this is an error condition. The newly added node should not receive any packets of the stream for which the newly added node is not the owner.

[0149] At 1740, the process determines whether the flow is a new flow assigned to the new node. Unlike existing flows that have existing state information stored in their original owner nodes, new flows do not have original owner nodes or pre-existing state information, and can therefore begin stateful processing at their owner nodes at will, whether a newly added node or an existing node. In some embodiments, the process determines whether the flow is a new flow by checking the entries in its new flow table. If the packetized flow is a newly added flow, the process proceeds to 1745. If the flow is not a newly added flow (i.e., a pre-existing flow), the process proceeds to 1750.

[0150] At 1750, the process determines whether the identified flow is in the indirect table of the edge node. In this operation, the process has determined that the newly added node running the process is the owner node of the flow identified by consistent hashing. However, the corresponding entry in the indirect table will indicate to the newly added node that the flow must be forwarded elsewhere for stateful processing. If the identified flow has a corresponding entry in the indirect table, the process proceeds to 1755. Otherwise, the process 1700 ends because the newly added node should not receive packets for existing flows without a corresponding entry in the indirect table.

[0151] At 1755, the process forwards the packet to a delegated node in the cluster for stateful processing (even if the newly added node running the process is the owner node of the flow). The process 1700 then ends.

[0152] Some embodiments allow multiple edge nodes to be added to the cluster simultaneously. In some of these embodiments, the cluster immediately enters a learning mode for multiple edge nodes while pinning the flow table and the indirection table are also used to delegate groups of flows anchored / pinned to their original owner nodes. Some embodiments add multiple edge nodes to the cluster by adding one edge node at a time, where multiple rounds of learning mode are used to add multiple new edge nodes.

[0153] b. Remove edge nodes

[0154] In some embodiments, the removal of edge nodes is done gracefully without interrupting stateful services provided by the edge cluster. For some of these embodiments, the method for adding new nodes described in Section II.a above also applies to dynamically resizing the edge cluster by removing existing nodes, i.e., maintaining stateful processing of existing flows at their original owner nodes during the node removal process. As mentioned, changes in membership in the edge cluster result in changes in the allocation of flows to nodes via consistent hashing. Therefore, the removal of a node causes existing flows to change owner nodes, just like the addition of a node. Therefore, some embodiments also anchor existing flows at their original owner nodes during the node removal process.

[0155] Fig.18 An example node removal operation is conceptually illustrated in which an existing flow is anchored to its original owner node. As shown, edge cluster 810 initially has nodes 811-814 (nodes A, B, C, and D), and the allocation of owner nodes for flows is based on consistent hashing of all four nodes. The figure illustrates a process for removing node C (813) from cluster 810 in two stages 1801 and 1802.

[0156] The first stage 1801 shows the allocation of owner nodes by consistent hashing before removing node C. As shown, before removing node C, the consistent hashing of the stream is on nodes A, B, C, and D to identify the owner node from among these four nodes. Under this consistent hashing function, streams F1 and F5 are assigned to node A (811), streams F2 and F6 are assigned to node B (812), streams F3 and F7 are assigned to node C (813), and streams F4 and F8 are assigned to node D (814). For this example, it is assumed that streams F1-F8 are active, ongoing streams that are statefully processed by nodes A, B, C, and D during stage 1801.

[0157] The second stage 1802 shows the removal of new node C and the impact of this removal on the owner node allocation by consistent hashing. With the removal of node C, consistent hashing is only on nodes A, B and D, so as to identify the owner node from among these three nodes. Therefore, some or all of streams F1-F8 are reallocated to different owner nodes. In this example, all streams except F5 and F6 are allocated to different owner nodes by consistent hashing. Node A is now the owner node of F3, F5 and F8. Node B is now the owner node of F1, F4 and F6. Node D is now the owner node of F2 and F7. The node (node ​​C) removed is not the owner node of any stream.

[0158] Since flows F1-F8 are active, ongoing flows being processed by the edge cluster, these nodes must be anchored to their original owner nodes so that the flows are processed statefully with the correct state information. Therefore, flow F1 is pinned to node A, flow F2 is pinned to node B, flows F3 and F7 are pinned to node C, and flows F4 and F8 are pinned to node D. Flows that are pinned or anchored to their original owner nodes will be processed statefully there until those flows are completed. This also applies to the removal of node C, where node C will remain active for the purpose of processing the flows pinned to node C (F3 and F7), even though node C can no longer be the owner node for any flows.

[0159] On the other hand, the new owner node of the existing flows delegates these flows to their original owners using the entries in its indirection table. Section II.a above describes delegating flows via the indirection table and the fixed flow table. In this example, node A delegates flows F3 and F8 to nodes C and D, respectively (F5 is processed locally at node A), node B delegates flows F1 and F4 to nodes A and D, respectively (F6 is processed locally at node B), and node D delegates F2 and F7 to nodes B and C, respectively.

[0160] As mentioned, the removed node will not go offline until it has completed processing of the flows previously assigned to it. This ensures that the node is removed or shut down gracefully without disrupting the stateful services provided by the edge cluster 810. In some embodiments, the edge cluster is considered to be in a learning mode for removing edge nodes in the cluster. In some embodiments, the learning mode for removing edge nodes will not end until the removed node has completed all of its fixed flows and can be safely offline (thus will not disrupt stateful processes).

[0161] In some embodiments, once a node is marked for removal, it stops participating in ECMP. Fig.19 The operation of edge cluster 810 during and after a learning mode for removing node 813 (i.e., learning node C to be removed) is shown. Once removed node 813 is marked for removal, it stops participating in ECMP, but the node does not go offline until all flows pinned to the removed node are completed. The figure shows the node removal process in four stages 1901-1904.

[0162] The first stage 1901 shows the edge cluster 810 before removing node C (813). As shown, the edge cluster 810 has four nodes 811-814 (nodes A, B, C, and D) that are participating in the ECMP process and providing stateful services. As shown, node 813 is the owner node of F3 and F7 through consistent hashing, and is maintaining the state information of these flows.

[0163] The second stage 1902 shows the edge cluster 810 immediately after node C (813) is marked for removal. Once node C is marked for removal, the ECMP process stops distributing packets to it. In addition, the consistent hashing no longer includes node C in its hash function, and node A (811) becomes the new owner node of flow F3, and node D becomes the new owner node of F7. However, F3 and F7 are also fixed or anchored to node C (813) because they are actively processed by node C and their state information is maintained by node C. Node A has an entry in its indirect table (not shown in the figure) for delegating the packets of flow F3 to node C, and node D has an entry in its indirect table (not shown in the figure) for delegating the packets of flow F7 to node C.

[0164] The third stage 803 shows the completion of the fixed flows (i.e., F3 and F7). When these flows are completed, the removal of node C (813) can be taken offline. The final stage 804 shows the cluster 810 in a new stable state, where node C is removed gracefully without interrupting any stateful services.

[0165] III. Related Flows

[0166] When the necessary state information is not in the owner node of the stream through consistent hashing, the delegation of the stream (or the transmission of state information) is necessary. As described in Part II above, this is the case when there is a membership change (e.g., the addition or removal of a node) in the edge cluster that changes the owner node assignment. In some embodiments, when the state information of the first stream residing on the first node is the necessary state information of the second stream assigned to the second node, the delegation of the stream is also required. In some embodiments, the first stream and the second stream are related streams assigned to different owner nodes through consistent hashing. In some embodiments, this occurs when the control stream and the data stream of the same process are assigned to different owner nodes because they have different stream identifiers. In order to ensure that the related streams operate on the same set of state information maintained on the same node, some embodiments use stream delegation to force the related streams to be processed by the same node (statefully), even if the two related streams are hashed to different owner nodes through consistent hashing.

[0167] There are protocols such as FTP, SIP, or RTSP that require ALG support to work through firewall and NAT functions. These protocols use multiple streams, and the same node should usually handle related streams because they often share state. Therefore, the data plane of the node needs to look at the application protocol to determine the related streams. In cluster services, related streams used in the protocols must be consistently and efficiently manipulated by the same node. For FTP (File Transfer Protocol), in some embodiments, a control flow is first established. Information about the data flow is transmitted via the control flow. Therefore, the control flow must be monitored to understand the data flow information (i.e., PORT or PASV commands on the control flow).

[0168] Fig. 20 The use of stream delegation to process related streams at the same node is shown. As shown, nodes 813 (node ​​C) and node 814 (node ​​D) of cluster 810 are assigned streams F15 and F16 respectively through consistent hashing. Streams F15 and F16 are related streams (e.g., control streams and data streams exchanged at the same transport layer). The figure shows related stream delegation in cluster 810 in two stages 2001 and 2002.

[0169] Stage 1 2001 shows the original allocation of F15 and F16 through consistent hashing. Node C, which is the owner node of F15, will have the state information of flow F15, and node D, which is the owner node of F16, will have the state information of flow F16. However, F15 and F16 are related flows. In some embodiments, related flows are flows that share the same set of state information. In some embodiments, related flows are flows that must be processed sequentially. For example, the stateful service of F16 requires the final state information of flow F15, and F16 is the flow after F15, so that the packet of flow F16 appears only after flow F15 is completed.

[0170] Second stage 2002 shows the delegation flow for processing the grouping of related streams.As shown in the figure, stream F15 is still processed by its owner node (node ​​C), but the grouping of stream F16 is delegated to node C by node D (owner node of F16).Node D forwards the grouping of F16 to node C based on the entry 2041 of indirect table 2040, and this entry corresponds to the stream F16 pointing to node C.Node C accepts the delegation grouping of stream F16 based on the entry 2031 of fixed stream table 2030.In other words, even if node D is the owner node of F16, stream F16 has also become the indirect / delegation flow at node D.Even if node C is never the owner node of F16, stream F16 has also become the stream fixed / anchored to node C.Therefore, stream F15 and F16 are both processed statefully by the same node (node ​​C).

[0171] For some embodiments, Fig.21 Conceptually illustrated is a process 2100 for establishing stream delegation so as to process related streams statefully in a cluster of edge nodes. In some embodiments, the process is performed by a node in an edge cluster. The process begins by determining (at 2110) whether the stream currently active on this edge node (the edge node running process 2100) has a related stream. For example, the control stream of FTP has a related data stream, so the edge node that processes the control stream of FTP will know that there is an upcoming related data stream. If the streams actively running on the edge node do not have a related stream, process 2100 ends. If at least one stream in the stream running in the edge node has a related stream, the process identifies (at 2120) the related stream and the owner node of the related stream. In some embodiments, the edge node identifies the related stream based on the information provided by the network manager or other edge nodes via a control plane message.

[0172] Next, the process creates (at 2130) an entry in its own fixed table for the relevant flow and sends (at 2140) a message to the owner node of the relevant flow to create an entry in its indirect table. Fig. 20In the example of , node C is an edge node that is processing flow F15, which has a related flow F16 whose owner node is node D. Therefore, node C creates a fixed flow entry (2031) for flow F16 in its fixed flow table (2030). Node C also sends a message to node D, so node D will create an indirect entry (2041) for flow F16 in its indirect table (2040).

[0173] After sending the message to establish the indirection entry, process 2100 ends and the edge nodes in the cluster continue to process incoming packets of the relevant flow based on the indirection table and the fixed flow table.

[0174] IV. Address Translation

[0175] In some embodiments, the stateful services provided by the edge cluster include network address translation (NAT). NAT is a method of modifying the network address information in Internet Protocol (IP) datagram packet headers as they are transmitted across traffic routing devices in order to remap one IP address space to another. In the case of DNAT, the destination address (e.g., often a VIP address / port) is replaced with a southbound address and port. This creates two flows associated with the DNAT service: one using the original destination address, and one using the translated address. These two flows are actually the same flow, but they still have different flow identifiers and will be hashed to different edge nodes. In other words, each flow has two owner nodes, one identified by the original destination address of the flow and the other identified by the translated address of the flow.

[0176] In some embodiments, the owner node of the flow according to the original address pushes an indirect message to the owner node of the flow according to the converted DNAT address so as to keep the packets of the flow on the same node. As a result, the owner node identified by the converted DNAT address has an indirect table entry for delegating the packets of the flow to the owner node identified by the original address, which has a fixed flow table entry that accepts and processes packets with flow identifiers based on the DNAT address. (In contrast, in some other embodiments, the owner node identified by the DNAT address pushes an indirect message to the owner node identified by the original address, so the owner node identified by the original address delegates the packets of the flow to the owner node identified by the DNAT address). In other words, in some embodiments, the flow as identified by its original address and the flow as identified by its DNAT address are processed as related flows by the edge cluster.

[0177] Fig.22aAn example DNAT service performed by edge cluster 810 is conceptually illustrated. The figure shows four example flows F1-F4 with different flow identifiers. The DNAT performed by edge cluster 810 converts the destination address of the southbound flow from its original address to the DNAT address. For example, edge cluster 810 converts the destination address of southbound F1 from "2.2.10.10" to "1.2.3.4", and converts the destination address of southbound F2 from "3.2.10.11" to "5.6.7.8", and so on.

[0178] DNAT also performs a reverse DNAT service for northbound traffic. The reverse DNAT service converts the DNAT address of the northbound flow back to its original address. For example, the edge cluster 810 converts the destination address of northbound F1 from "1.2.3.4" to "2.2.10.10", converts the destination address of northbound F2 from "5.6.7.8" to "3.2.10.11", and so on. Although the northbound flow and the southbound flow are the same flow, they have different flow identifiers and will be hashed to different edge nodes within the cluster 810 through consistent hashing.

[0179] Figure 22b 810 is shown how owner nodes are assigned when nodes in edge cluster 810 are performing DNAT service. As shown, for southbound traffic where the flow identifier is based on the original address, flows F1-F4 are hashed to nodes A, B, C, and D (nodes 811-814), respectively. However, for northbound traffic where the flow identifier is based on the converted DNAT address (i.e., DNAT(F1), DNAT(F2), DNAT(F3), and DNAT(F4)), flow F1 is hashed to node D, flow F2 is hashed to node B, flow F3 is hashed to node A, and flow F4 is hashed to node C. In the figure, "Hash(F1)" means that F1 is hashed based on the flow identifier with the original destination address, while "Hash(DNAT(F1))" means that F1 is hashed based on the flow identifier with the converted destination address.

[0180] Figure 22bIt is also shown that the edge cluster 810 uses flow delegation to provide a stateful DNAT service. As shown, node A sends an indirect message for DNAT (F1) to node D, notifying node D to delegate the packet with the flow identifier to node A based on DNAT (F1). Node C sends an indirect message for DNAT (F3) to node A, notifying node A to delegate the packet with the flow identifier to node C based on DNAT (F3). Node D sends an indirect message for DNAT (F4) to node C, notifying node C to delegate the packet with the flow identifier to node D based on DNAT (F4). Both F2 and DNAT (F2) are hashed to node B, so node B does not send indirect messages outward.

[0181] Thus, node A receives a delegation packet with DNAT(F1) so that node A can statefully process all packets of flow F1, both northbound and southbound. Likewise, node C receives a delegation packet with DNAT(F3) to statefully process all packets of flow F3, and node D receives a delegation packet with DNAT(F4) to statefully process all packets of flow F4. (In this example, node B has received packets for both F2 and DNAT(F2)).

[0182] In some other embodiments, instead of flow delegation, the edge cluster performing DNAT determines the owner node without using the destination address that can be translated. In other words, the cluster only applies consistent hashing to fields that are not affected by DNAT (e.g., source address field, source port field, protocol field). Fig.23a -b shows an edge cluster that does not use the destination address field for node allocation when performing DNAT.

[0183] Fig.23a Southbound traffic is shown where the destination address is DNAT'ed from its original address to the translated address. Southbound packet 2311 is a packet of flow F1, packet 2312 is a packet of flow F2, packet 2313 is a packet of flow F3, and packet 2314 is a packet of flow F4. Packets 2311-2314 are hashed to nodes 811-814 of edge cluster 810, respectively. The hash is based only on the source address and source port of the packet. The destination addresses of those packets are not used when applying consistent hashing to determine the owner node of the flow. The nodes in edge cluster 810 perform DNAT services on those packets to generate packets 2321-2324. These packets have the same original source address, but their destination addresses are addresses translated from DNAT.

[0184] Figure 23bNorthbound traffic is shown in which the destination address is returned from its converted address to its original address. Northbound packets 2331-2334 are packets of flows F1-F4, respectively. The destination addresses of these northbound packets are addresses converted by DNAT, but their source addresses and ports are original addresses that are not affected by DNAT. Packets 2331-2334 are hashed to nodes A, B, C, and D based on those unaffected source ports and addresses. Therefore, the packets of flows F1-F4 are still processed by nodes A, B, C, and D, respectively, and the DNAT has no effect on the owner node allocation. Therefore, regardless of whether the destination address field is converted or original, each flow is processed statefully by the same node.

[0185] In the case of SNAT, the edge nodes in the cluster change the source address of outgoing southbound packets. SNAT of the packet occurs after routing through the owner node of its flow. In some embodiments, each node in the edge cluster uses its own unique SNAT address when performing address translation. Based on those unique SNAT addresses in the source address field, the returning northbound traffic is forwarded to the same owner node. The owner node then applies reverse NAT while maintaining the state of the stateful SNAT process. If there is a firewall service, the flow state is identified by the tuple after the SNAT reverse mapping.

[0186] Fig.24a -b shows how the nodes in the edge cluster provide stateful SNAT services for some embodiments. As shown, the edge nodes 811-814 (nodes A, B, C and D) of cluster 810 are providing SNAT services, wherein when performing SNAT, each node has its own unique source address. Specifically, the SNAT address of node A is "1.1.1.1", the SNAT address of node B is "2.2.2.2", the SNAT address of node C is "3.3.3.3", and the SNAT address of node D is "4.4.4.4".

[0187] Fig.24a Southbound traffic packets 2411-2414 belonging to flows F1, F2, F3, and F4, respectively, are shown. The packets of flows F1-F4 are hashed into nodes A, B, C, and D according to the consistent hashing of the flow identifiers of the packets. Node A performs SNAT on the packets belonging to F1 in turn and converts the source address to its own SNAT address "1.1.1.1". Similarly, node B performs SNAT on the F2 packet and converts the source address to its own "2.2.2.2", node C performs SNAT on the F3 packet and converts the source address to its own "3.3.3.3", and node D performs SNAT on the F4 packet and converts the source address to its own "4.4.4.4".

[0188] Figure 24b It is shown that the northbound traffic is returned to the cluster 2410. As shown, each packet in the northbound traffic finds its path back to the edge node that performs SNAT based on the unique SNAT address in the source address field. Specifically, the northbound packet 2431 of F1 arrives at node A based on the source address "1.1.1.1", the northbound packet 2432 of F2 arrives at node B based on the source address "2.2.2.2", the northbound packet 2433 of F3 arrives at node C based on the source address "3.3.3.3", and the northbound packet 2434 of F4 arrives at node D based on the source address "4.4.4.4". Each node performs reverse SNAT on the northbound packet by identifying the flow of the packet and converting the unique SNAT address of the node back to the original source address of the flow. Therefore, the packets of streams F1-F4 are always processed by nodes A, B, C and D respectively, and SNAT has no effect on the owner node allocation. Regardless of whether the source address field is converted or original, each flow is processed statefully by the same node.

[0189] V. Virtual Machines as Edge Nodes

[0190] As mentioned, in some embodiments, some or all of the edge nodes in the edge cluster are implemented as virtual machines hosted by computing devices running virtualization software or hypervisors. In some embodiments, these computing devices (or host machines) are located in a data center that houses many such host machines. Fig.25 A data center 2500 is shown with host machines grouped into clusters or sections (clusters 2510, 2520, 2530, and 2540). Each cluster includes one or more host machines that are hosting virtual machines. Some of the virtual machines are computing VMs that perform computing tasks for clients or tenants of the data center. Some of the virtual machines are edge VMs that provide edge services such as NAT or firewalls. Some of the virtual machines are management VMs that provide control to other VM / host machines in the data center. Some host machines host different types of VMs. For example, host machine 2511 hosts computing VMs and management VMs, and host machine 2512 hosts computing VMs and edge VMs.

[0191] In some embodiments, each cluster or portion of VMs has its own set of edge service VMs that serve as its edge cluster, which provides stateful services in the manner described in Sections I-IV above.

[0192] In some embodiments, the data center has a specialized cluster or portion, such as a compute cluster or a management cluster. In some embodiments, the data center includes a cluster or portion of host machines that primarily hosts edge service VMs that serve as edge nodes for compute VMs in other clusters. Fig.26A data center 2600 is shown including an edge cluster 2610, several compute clusters 2651-2652, and a management cluster 2680. The edge cluster 2610 hosts VMs that serve as edge nodes for VMs in the compute clusters 2651-2652. The VMs in the management cluster 2680 perform control and management functions.

[0193] Regardless of the physical topology and actual physical placement of the VMs, in some embodiments, the compute VMs are logically organized into different logical networks for different tenants, and the edge service VMs are logically organized into edge service clusters for different logical networks. Fig. 27 Logical networks for different tenants are shown, where each tenant is served by a set of edge service VMs that serve as an edge service cluster for the logical network. The figure shows logical networks 2701-2703 for tenants 1, 2, and 3, respectively. Logical network 2701 uses edge service cluster 2711, logical network 2702 uses edge service cluster 2712, and logical network 2703 uses edge service cluster 2713. Each edge service cluster includes multiple edge service VMs as edge nodes. In some embodiments, the edge nodes of the edge cluster are in the same L2 segment, so that the packet forwarding and flow delegation operations of the edge cluster are performed through L2 layer traffic (i.e., edge nodes in the same edge cluster communication can address each other through L2 MAC addresses).

[0194] VI. Computing Equipment

[0195] As mentioned previously, some embodiments of the present invention are implemented by virtualization software or a hypervisor running on a computing device used as a host machine. Fig.28 An example host machine 2800 is shown operating virtualization software 2805. Virtualization software 2805 allows the host machine to host virtual machines 2811-2814 and connect the virtual machines to a physical network 2890. The physical network 2890 may span one or more data centers and include various physical switches and routers.

[0196] As shown, host 2800 is able to access physical network 2890 through physical NIC (PNIC) 2895. Virtualization software 2805 serves as an interface between hosted VMs 2811-2814 and physical NIC 2895 (as well as other physical resources, such as processors and memory). Each VM includes a virtual NIC (VNIC) for accessing the network through virtualization software 2805. Each VNIC in the VM is responsible for exchanging packets between the VM and virtualization software 2805. In some embodiments, a VNIC is a software abstraction of a physical NIC implemented by a virtual NIC emulator.

[0197] Virtualization software 2805 manages the operation of VMs 2811-2814 and includes several components for managing VM access to physical networks (in some embodiments, by implementing a logical network to which the VM is connected). As shown, virtualization software 2805 includes a physical switching element 2820, a physical routing element 2830, a controller interface 2840, and an uplink module 2870.

[0198] Controller interface 2840 receives control plane messages from a controller or controller cluster 2860. In some embodiments, these control plane messages include configuration data for configuring various components of the virtualization software and / or virtual machines, such as physical switching elements 2820 and physical routing elements 2830. In some embodiments, control plane messages also include messages from a central network manager or a local network manager for dynamically adjusting the size of edge clusters.

[0199] The physical switching element 2820 (or managed physical switching element MPSE) passes network data to and from the physical NIC 2895 that interfaces with the physical network 2890. The physical switching element also includes a plurality of virtual ports (vPorts) that communicatively interconnect the physical NIC with the VMs 2811-2814, the physical routing element 2830, and the controller interface 2840. In some embodiments, each virtual port is associated with a unique L2 MAC address. The physical switching element performs L2 link layer packet forwarding between any two network elements connected to its virtual ports. The physical switching element also performs L2 link layer packet forwarding between any network element connected to any one of its virtual ports and an L2 network element reachable on the physical network 2890 (e.g., another VM running on another host).

[0200] The physical routing element 2830 (or the managed physical routing element MPRE) performs L3 routing (e.g., by performing L3 IP address to L2 MAC address resolution) on the data packets received from the virtual port on the physical switching element 2820. In some embodiments, the virtual port to which the physical routing element 2830 is attached is a sink port. Each routed data packet is then sent back to the physical switching element 2820 to be forwarded to its destination according to the resolved L2 MAC address. The destination may be another VM connected to a virtual port on the physical switching element 2820, or a L2 network element reachable on the physical network 2890 (e.g., another VM running on another host, a physical non-virtualized machine, etc.).

[0201] The uplink module 2870 relays data between the physical switching element 2820 and the physical NIC 2895. In some embodiments, the uplink module 2870 allows the host machine 2800 to be used as a tunnel endpoint for encapsulated overlay networks such as VXLAN and VLAN. VXLAN is an overlay network encapsulation protocol. The overlay network created by VXLAN encapsulation is sometimes referred to as a VXLAN network, or simply VXLAN. When a VM on the host 2800 sends a data packet (e.g., an Ethernet frame) to another VM in the same VXLAN network but on a different host, the uplink module 2870 uses the VNI of the VXLAN network and the network address of the VTEP to encapsulate the data packet before sending the packet to the physical network. The packet is tunneled to the destination host through the physical network (i.e., the encapsulation makes the underlying packet transparent to the intermediate network element). The uplink module 2870 also decapsulates the incoming VXLAN packet and only forwards the original internal data packet to the destination VM.

[0202] Fig.29 An edge service VM 2900 (e.g., as one of VMs 2811-2814) is shown running in a host machine 2800. The edge VM 2900 exchanges data plane packets with a physical switching element 2820 and exchanges control plane messages with a controller interface 2840. As shown, the edge service VM 2900 includes a stateful service engine 2910, a packet forwarding engine 2920, a consistent hashing engine 2930, and a flow information store 2940.

[0203] The stateful edge service engine 2910 is used to provide stateful edge services (e.g., firewall, NAT) based on the incoming packets from the physical switching packet and the state of the flow stored in the flow information storage 2940. In some embodiments, the stateful service engine 2910 provides the necessary data to the packet to the forwarding engine 2920.

[0204] The forwarding engine 2920 receives and forwards network packets via the physical switching element 2820. In some embodiments, the forwarding engine makes its forwarding decisions or changes packets based on information provided by the stateful edge service engine 2910, such as address translation or firewall decisions. In some embodiments, the forwarding engine 2920 sends the flow identifier of each incoming packet to the consistent hashing engine 2930 to determine its owner node. The forwarding engine 2920 also uses information in the flow information storage 2940 to decide whether to forward the packet or where to forward the packet. In some embodiments, the forwarding engine 2920 performs processes such as 700, 900, 1500, 1600, and 1700.

[0205] The consistent hash engine 2930 performs a consistent hash function on the flow identifier of the packet to identify its owner node. In some embodiments, the hash result is also based on the membership of the edge cluster provided by the network manager via the controller interface 2830 (e.g., the number of edge nodes in the cluster).

[0206] In some embodiments, the flow information storage 2940 stores tables required for implementing flow delegation, such as a fixed flow table 2950 and an indirect table 2955. In some embodiments, the flow information storage 2940 also includes a local flow table 2960, which maintains state information of the flow being processed by the edge VM. In some embodiments, the fixed flow table 2950 and the local flow table 2960 are implemented as a table that tracks all flows being processed by the edge VM 2900. The stored state is provided to the stateful service engine 2910 for performing stateful services.

[0207] In this document, the term "packet" refers to a collection of bits in a specific format sent across a network. Those of ordinary skill in the art will recognize that the term packet can be used herein to refer to various formatted collections of bits that can be sent across a network, such as Ethernet frames, TCP segments, UDP datagrams, IP packets, etc.

[0208] Reference is made throughout this specification to computing and network environments including virtual machines (VMs). However, a virtual machine is only one example of a data computing node (DCN) or a data computing end node (also referred to as an addressable node). A DCN may include non-virtualized physical hosts, virtual machines, containers that run on top of a host operating system without the need for a hypervisor or a separate operating system, and a hypervisor kernel network interface module.

[0209] In some embodiments, the VM uses the resources of the host virtualized by the virtualization software (e.g., a hypervisor, a virtual machine monitor, etc.) to operate with its own client operating system on the host. The tenant (i.e., the owner of the VM) can choose which applications to operate on top of the client operating system. On the other hand, some containers are structures that run on top of the host operating system without the need for a hypervisor or a separate client operating system. In some embodiments, the host operating system uses a namespace to isolate containers from each other, and thus provides operating system-level isolation of different application groups operating in different containers. This isolation is similar to the VM isolation provided in the hypervisor virtualization environment of the virtualized system hardware, and can therefore be regarded as a form of virtualization that isolates different application groups operating in different containers. This container is lighter than a VM.

[0210] In some embodiments, the hypervisor kernel network interface module is a non-VM DCN that includes a network stack with a hypervisor kernel network interface and receive / transmit threads. An example of a hypervisor kernel network interface module is an ESXi host as part of VMware Inc. TM The vmknic module is part of the hypervisor.

[0211] Those skilled in the art will recognize that although this specification refers to VMs, the examples given may be any type of DCN, including physical hosts, VMs, non-VM containers, and hypervisor kernel network interface modules. In fact, in some embodiments, the example network may include a combination of different types of DCNs.

[0212] VII. Electronic Systems

[0213] Many of the above features and applications are implemented as software processes, which are specified as a set of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more processing units (e.g., one or more processors, processor cores, or other processing units), they cause the (one or more) processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, RAM chips, hard drives, EPROMs, etc. Computer-readable media do not include carrier waves and electronic signals transmitted wirelessly or through wired connections.

[0214] In this specification, the term "software" refers to applications stored in magnetic storage including firmware residing in a read-only memory or that can be read into a memory for processing by a processor. In addition, in some embodiments, several software inventions can be implemented as sub-parts of a larger program while maintaining distinct software inventions. In some embodiments, several software inventions can also be implemented as separate programs. Finally, any combination of separate programs that together implement the software inventions described herein is within the scope of the present invention. In some embodiments, when the software program is installed to operate on one or more electronic systems, the software program defines one or more specific machine implementations that perform the operations of the software program.

[0215] Fig.30The electronic system 3000 that implements some embodiments of the present invention is conceptually shown. The electronic system 3000 can be used to perform any control, virtualization or the above-mentioned operating system application. The electronic system 3000 can be a computer (e.g., a desktop computer, a personal computer, a tablet computer, a server computer, a mainframe, a blade computer, etc.), a phone, a PDA or any other type of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 3000 includes a bus 3005, (one or more) processing units 3010, a system memory 3025, a read-only memory 3030, a permanent storage device 3035, an input device 3040, and an output device 3045.

[0216] Bus 3005 collectively represents all system, peripheral and chipset buses that communicatively connect numerous internal devices of electronic system 3000. For example, bus 3005 communicatively connects (one or more) processing units 3010 with read-only memory 3030, system memory 3025, and permanent storage device 3035.

[0217] From these various memory units, the processing unit(s) 3010 retrieve instructions to execute and data to process in order to perform the processes of the present invention. The processing unit(s) may be a single processor or a multi-core processor in different embodiments.

[0218] Read-only memory (ROM) 3030 stores static data and instructions required by (one or more) processing units 3010 and other modules of the electronic system. On the other hand, permanent storage device 3035 is a read and write memory device. This device is a non-volatile storage unit that stores instructions and data even when the electronic system 3000 is turned off. Some embodiments of the present invention use a mass storage device (such as a magnetic or optical disk and its corresponding disk drive) as permanent storage device 3035.

[0219] Other embodiments use removable storage devices (such as floppy disks, flash drives, etc.) as permanent storage devices. Like permanent storage device 3035, system memory 3025 is a read and write memory device. However, unlike storage device 3035, system memory is a volatile read and write memory, such as random access memory. System memory stores some instructions and data that the processor needs at run time. In some embodiments, the processes of the present invention are stored in system memory 3025, permanent storage device 3035, and / or read-only memory 3030. From these various memory units, (one or more) processing unit 3010 retrieves instructions to be executed and data to be processed in order to perform the processes of some embodiments.

[0220] The bus 3005 is also connected to input and output devices 3040 and 3045. Input devices enable a user to pass information and select commands to the electronic system. Input devices 3040 include an alphanumeric keyboard and a pointing device (also known as a "cursor control device"). Output devices 3045 display images generated by the electronic system. Output devices include printers and display devices, such as cathode ray tubes (CRTs) or liquid crystal displays (LCDs). Some embodiments include devices such as touch screens that function as both input and output devices.

[0221] Finally, as in Fig.30 As shown in FIG. 30 , bus 3005 also couples electronic system 3000 to a network 3065 via a network adapter (not shown). In this manner, the computer may be part of a network of computers such as a local area network ("LAN"), a wide area network ("WAN"), or an intranet, or a network such as the Internet. Any or all components of electronic system 3000 may be used in conjunction with the present invention.

[0222] Some embodiments include electronic components such as microprocessors, storage devices and memories that store computer program instructions in machine-readable or computer-readable media (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, compact disk-read only (CD-ROM), compact disk-recordable (CD-R), compact disk-rewritable (CD-RW), read-only digital versatile disks (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini SD card, micro SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable A computer readable medium may store a computer program that is executable by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer code include machine code such as produced by a compiler, and files including higher level code that is executed by a computer, electronic component, or microprocessor using an interpreter.

[0223] Although the above discussion mainly refers to a microprocessor or multi-core processor that executes software, some embodiments are implemented by one or more integrated circuits, such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). In some embodiments, such an integrated circuit executes instructions stored on the circuit itself.

[0224] As used in this specification, the terms "computer", "server", "processor", and "memory" all refer to electronic or other technical devices. These terms do not include people or groups of people. For the purposes of this specification, the terms display or being displayed mean displayed on an electronic device. As used in this specification, the terms "computer-readable medium", "multiple computer-readable media" and "machine-readable medium" are completely limited to tangible, physical objects that store information in a form that is readable by a computer. These terms do not include any wireless signals, wired download signals, and any other transient signals.

[0225] Although the present invention has been described with reference to many specific details, those skilled in the art will recognize that the present invention may be embodied in other specific forms without departing from the spirit of the present invention. Figure 7 , 9 , 15, 16, 17) conceptually illustrate the processes. The specific operations of these processes may not be performed in the exact order shown and described. Specific operations may not be performed in a series of continuous operations, and different specific operations may be performed in different embodiments. In addition, a process can be implemented using several sub-processes or as part of a larger macro process. Therefore, it will be understood by those skilled in the art that the present invention is not limited by the above illustrative details, but is defined by the appended claims.

Claims

1. A method for performing a stateful service for a network comprising a plurality of host computers, each host computer executing a set of virtual machines, wherein different sets of virtual machines on different hosts are connected via different logical networks, the method comprising: At the first edge service node in the cluster of edge service nodes specified for the logical network: performing a stateful service requiring state information of a corresponding flow in network traffic entering or leaving the logical network, wherein the first edge service node is selected from the cluster of edge service nodes based on a first hash of an identifier of the corresponding flow; receiving packets belonging to the corresponding flow from a second edge service node, wherein after at least one edge service node is added to or removed from the cluster, selecting the second edge service node from the cluster, the selecting being based on a second hash of an identifier of the corresponding flow; and The stateful service is performed on the received packet at the first edge service node.

2. The method according to claim 1, wherein: The corresponding edge service node in the cluster is executed on the same host computer as the virtual machine that is the source or destination of the corresponding traffic.

3. The method according to claim 1, wherein: Prior to adding or removing the at least one edge service node, a first mapping maps possible values ​​of the first hash to edge service nodes in the cluster; A second mapping maps possible values ​​of the second hash to edge service nodes in the cluster after adding or removing the at least one edge service node; and The second mapping is different from the first mapping.

4. The method according to claim 1, wherein: The first edge service node maintains state information of the corresponding flow, the state information being maintained for the corresponding flow based on performing a stateful service on a previous packet of the corresponding flow at the first edge service node.

5. The method according to claim 1, wherein: The cluster of edge service nodes provides stateful services at the boundary of the logical network.

6. The method according to claim 1, wherein: receiving the packet from the second edge serving node based on an entry in an indirection table of the second edge serving node, the entry associating the corresponding traffic with the first edge serving node; and The first edge service node has a fixed flow table, the fixed flow table includes entries corresponding to the corresponding traffic, and the entries are used to prevent the first edge service node from forwarding packets belonging to the corresponding traffic to other edge service nodes in the cluster.

7. The method according to claim 6, further comprising: When the stateful processing of the corresponding traffic is completed, the entry is removed from the fixed flow table.

8. A method for executing a stateful service for a network comprising a plurality of host computers, each host computer executing a set of virtual machines, wherein different sets of virtual machines on different hosts are connected via different logical networks, the method comprising: performing, at a cluster of edge service nodes, a stateful service requiring state information of network traffic entering or leaving a logical network, wherein a state of a respective one of the network traffic is maintained by a first edge service node in the cluster; adding an edge service node to the cluster while the cluster performs the stateful service for the corresponding traffic; forwarding packets belonging to the corresponding flow from the second edge service node to the first edge service node upon initially identifying a second edge service node to perform the stateful service via a consistent hashing function of a flow identifier of the corresponding flow of the cluster of edge service nodes used by each node in the cluster after adding the edge service node; and The stateful service is performed on the forwarded packet at the first edge service node.

9. The method according to claim 8, wherein: The first edge service node is identified by a different consistent hashing function of flow identifiers of the corresponding traffic of the cluster via edge service nodes used by each node in the cluster prior to adding the edge service node to perform the stateful service.

10. The method according to claim 8, wherein: Adding the edge service node to the cluster of edge service nodes further comprises: adding an entry for the corresponding traffic to an indirection table in the second edge service node, wherein the entry added in the indirection table is used to instruct the second edge service node to forward packets belonging to the corresponding traffic to the first edge service node; and An entry for the corresponding flow is added to a fixed flow table in the first edge service node, wherein the entry for the corresponding flow added to the fixed flow table is used to anchor packets of the corresponding flow at the first edge service node.

11. The method according to claim 8, wherein: The second edge service node is an edge service node added to the cluster.

12. A method for statefully executing a service in a node cluster comprising a first node and a second node, the method comprising: At the first node: receiving a first packet of a first flow in the network traffic based on a first set of flow identifiers of the first flow; performing services on the first group and storing state information; receiving a second packet of a second flow in the network flow from the second node, the second flow being part of a connection session with the first flow, wherein the second node initially receiving the second packet based on a second set of flow identifiers for the second flow and then forwarding the second packet to the first node when the first node previously processed the first flow as part of the same connection session as the second flow; and A service is performed on the second packet by using state information stored at the first node for the connection session.

13. The method according to claim 12, wherein: The service performed for the first traffic is network address translation processing.

14. The method according to claim 13, wherein: The network address translation process translates the first set of flow identifiers for the first traffic into the second set of flow identifiers for the second traffic.

15. The method of claim 12, wherein: The first node is identified by a hash of the first set of flow identifiers and the second node is identified by a hash of the second set of flow identifiers.

16. The method according to claim 12, wherein: The second node stores an indirection table having entries for forwarding packets having the second set of flow identifiers from the second node to the first node, and the method further comprises using a fixed flow table having entries for maintaining and processing packets having the second set of flow identifiers at the first node.

17. The method according to claim 12, wherein: The hash of the flow identifier initially identifies an owner node of the flow, wherein the first node is the owner node of the first flow based on the hash of the first set of flow identifiers, and the second node is the owner node of the second flow based on the hash of the second set of flow identifiers.

18. A method for statefully executing a service at a node cluster comprising a plurality of nodes, the method comprising: At a first node among the plurality of nodes: Based on the original source address in the headers of multiple outgoing packets leaving the logical segment, receiving the plurality of outgoing packets; performing a service on each of the plurality of outgoing packets; as well as For each of the multiple outgoing packets, the original source address in the header is converted into a converted source address uniquely associated with the first node to ensure that incoming packets entering the logical network segment in response to the multiple outgoing packets are received by the first node based on the converted source address, rather than by another node in the node cluster.

19. The method according to claim 18, further comprising: An inverse source network address translation is performed on each incoming packet to replace the translated source address with the original source address of the corresponding network traffic.

20. The method of claim 18, further comprising: State information for the incoming packets is maintained.

21. A method for performing a stateful service for a network including a plurality of host computers, the method comprising: At each of a plurality of service virtual machine edge clusters serving a corresponding plurality of logical overlay networks: performing a set of stateful services on packets of network traffic belonging to a source computing node outside a corresponding logical overlay network to a destination virtual machine inside the corresponding logical overlay network, wherein each network traffic associated with a logical overlay network is assigned a service virtual machine in a service virtual machine edge cluster corresponding to the associated logical overlay network, the assigned service virtual machine being configured to perform the set of stateful services based on state information related to a state of the network traffic maintained at the assigned service virtual machine; as well as After performing the set of stateful services on each packet, providing the corresponding packet to the destination virtual machine for the network traffic.

22. The method of claim 21, wherein: At least one service virtual machine executes on the same host computer as a virtual machine that is a destination for at least one network traffic; and Each host computer on which the service virtual machine executes also executes a managed physical switching element and a managed physical routing element that, together with the managed physical switching elements and managed physical routing elements executed on other host computers, implement at least one logical overlay network.

23. The method according to claim 21, wherein: Each packet is assigned to a service virtual machine in the cluster based on (i) identifying the network flow to which the packet belongs based on a set of flow identifiers in the packet, and (ii) calculating a hash value based on the set of flow identifiers.

24. The method according to claim 23, wherein: Each possible hash value is mapped to a service virtual machine in the cluster.

25. The method according to claim 21, wherein: The set of stateful services includes at least one of network address translation and firewall processing for a stateful firewall.

26. The method according to claim 21, wherein: Each service virtual machine in the cluster is a virtual machine provided by virtualization software executed in a host computer among the plurality of host computers.

27. A method for providing a plurality of stateful services for a plurality of network flows based on state information relating to a state of each flow of a network including a plurality of host computers, the plurality of network flows comprising a first group of flows and a second group of flows, the method comprising: providing a plurality of stateful services at an edge cluster of service virtual machines for a logical overlay network, wherein a first stateful service is provided by a first set of service virtual machines in the cluster and a second stateful service is provided by a second set of service virtual machines in the cluster; allocating the first group of traffic to the first group of service virtual machines according to a first consistent hash function, the first group of traffic being between a first group of computing nodes outside the logical overlay network and a first group of virtual machines in the logical overlay network and requiring the first stateful service, wherein state information of corresponding traffic in the first group of traffic is maintained at the first service virtual machine to which the corresponding traffic is allocated according to the first consistent hash function; as well as The second group of traffic is allocated to the second group of service virtual machines according to a second consistent hash function, the second group of traffic is between a second group of computing nodes outside the logical overlay network and a second group of virtual machines in the logical overlay network, and requires the second stateful service.

28. The method of claim 27, wherein allocating the first group of traffic to the first group of service virtual machines comprises: (i) identifying an incoming packet belonging to the first group of traffic, wherein the incoming packet is a packet entering a logical network segment; and (ii) applying the first consistent hashing function to flow identifiers of the incoming packets identified as belonging to the first group of traffic.

29. The method according to claim 28, wherein: A first consistent hash function applied to a first flow generates a hash value, wherein the hash value identifies a first service virtual machine in a first group of service virtual machines to which the first flow is assigned, wherein a second consistent hash function applied to a second flow generates a hash value, wherein the hash value identifies a second service virtual machine in a second group of service virtual machines to which the second flow is assigned, wherein the service virtual machine to which the corresponding flow is assigned maintains the state of the corresponding flow.

30. The method of claim 27, wherein: Each possible hash value of the first consistent hash is mapped to a service virtual machine in the first set of service virtual machines, and each possible hash value of the second consistent hash is mapped to a service virtual machine in the second set of service virtual machines.

31. The method of claim 27, wherein: The first stateful service is network address translation and the second stateful service is a firewall.

32. A machine-readable medium storing a program which, when executed by at least one processing unit, implements the method according to any one of claims 1 to 31.

33. An electronic device comprising: a set of processing units; and A machine-readable medium storing a program which, when executed by at least one of the processing units, implements the method according to any one of claims 1 to 31.

34. An electronic system comprising means for implementing the method according to any one of claims 1-31.

35. A computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 31.

Images