Providing modular network services through distributed elastic middleboxes

Abstract

A distributed platform is provided for developing and deploying complex, resilient, highly available, multi-tenant network services. The network service system includes an in-network key / value store that implements a distributed flow table (DFT), the DFT including nodes organized into a chain. Each node in the chain includes a modular middlebox that is linked to at least one other modular middlebox in the chain to implement network service functions for data packets based on a specification table to generate flow table entries. The DFT scales horizontally and maintains consistent connection state between replicas. A stateless forwarding layer performs data packet forwarding decisions for established connections and bounces data packets for transmission. A management node converts network services described in a declarative programming language into the specification table for distribution to the modular middleboxes in the nodes of the DFT.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 037,479, filed on June 10, 2020, entitled “Modular Network Services Via Distributed Elastic Middleboxes,” the contents of which are incorporated herein by reference. Technical Field

[0003] The present application relates to a distributed platform for developing and deploying multi-tenant network services, and more particularly to a system and method for sharing cloud resources across multi-tenant network services and gateways while creating and maintaining a distributed shared state. Background Art

[0004] A middlebox is a computer network device used for purposes other than packet forwarding, such as translating, inspecting, filtering, and manipulating data traffic. Examples of middleboxes include firewalls, network address translators (NATs), load balancers, and deep packet inspection (DPI) boxes. Existing software-based middleboxes or virtual switches, such as ClickOS (described by Martins et al. in "ClickOS and the Art of Network Function Virtualization," 11th USENIX Symposium on Network Systems Design and Implementation (NSDI 14), pp. 459–473, 2014) and Open vSwitch (OVS), (described by Pfaff et al. in "The Design and Implementation of Open vSwitch," 12th USENIX Symposium on Network Systems Design and Implementation (NSDI 15), pp. 117–130, 2015), maintain local shared connection state but lack mechanisms for sharing state information. Adding or removing middleboxes without maintaining consistent state across multiple middleboxes during failover or scale-out can lead to connectivity loss and inconsistent packet forwarding decisions.

[0005] Hardware-based middleboxes are expensive, non-scalable, and difficult to operate and scale in cloud environments. On the other hand, deploying existing software-based middleboxes in virtual machines or containers to replace hardware gateways can lead to underutilized resources, cost inefficiencies, and a lack of resource sharing across multiple tenants and different network functions. Furthermore, models like ClickOS and OVS impose significant limitations on service availability and scalability. Summary of the Invention

[0006] Various examples are now described to introduce some concepts in a simplified form that will be further described in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0007] In an exemplary embodiment, an architecture is provided for a platform that decouples network service scaling, programming, maintenance, upgrades, configuration, and failure domains from data center networks. The platform enables sharing of cloud resources, including compute and network resources, across multi-tenant network services and gateways. The platform also provides techniques for creating and maintaining distributed shared state, including but not limited to common configuration, flow state, and port allocations.

[0008] According to a first aspect of the present invention, a network service system is provided, comprising a connection state tracking layer, a stateless forwarding layer, and nodes for converting network services. The connection state tracking layer comprises first nodes organized into at least one chain, each first node in the chain comprising a modular middlebox, the modular middlebox linked to at least one other modular middlebox in the chain to implement a network service function for a data packet. The connection state tracking layer implements a distributed flow table (DFT) comprising the at least one chain of first nodes. The modular middlebox comprises at least one specification table that defines a sequence of actions for implementing the network service function for the data packet. The stateless forwarding layer comprises second nodes, each of which comprises at least one second node. The at least one second node receives the data packet, selects a flow entry from the DFT when available, rewrites the header of the data packet according to the flow entry, and sends the data packet to a first destination based on the flow entry. At least one third node converts the network service described in a declarative programming language into the at least one specification table and distributes the at least one specification table to at least one modular middlebox in the at least one first node chain of the DFT.

[0009] According to a second aspect of the present invention, a method for implementing a network service is provided. The method comprises: converting a network service described in a declarative programming language into at least one specification table; distributing the at least one specification table to modular middleboxes in nodes of a connection state tracking layer, the connection state tracking layer comprising a first node organized into at least one chain, the modular middleboxes linked to at least one other modular middleboxes to implement a network service function for a data packet, the connection state tracking layer implementing a distributed flow table (DFT) comprising the at least one first node chain; processing the data packet according to an action sequence specified in the at least one specification table to implement the network service function for the data packet, thereby generating a flow table entry; and sending the data packet to a destination based on the flow table entry.

[0010] In a first implementation of any of the above aspects, the DFT is implemented in a key / value memory.

[0011] In a second implementation of any of the above aspects, the modular middlebox includes a plurality of express data path (XDP) programs and a shared data structure configuration in a local data structure for processing data packets to implement the network service function.

[0012] In a third implementation of any of the above aspects, each first node in the at least one chain executes a chain replication protocol, which replicates state and data packet flow table entries to maintain a consistent flow state on all first nodes in the at least one chain.

[0013] In a fourth implementation of any of the above aspects, the at least one third node provides an extended endpoint operator that generates the specification table from an extended endpoint configuration written in the declarative programming language, and the extended endpoint operator further propagates the specification table of the network service function to a middlebox of the at least one first node chain of the DFT to implement the network service function.

[0014] In a fifth implementation manner of any of the above aspects, an extended endpoint is provided, which spans multiple second nodes of the stateless forwarding layer and is automatically expanded according to data packet traffic requirements.

[0015] In a sixth implementation manner of any one of the above aspects, a selector is provided, which maps the extended endpoint to at least one second node of the stateless forwarding layer.

[0016] In a seventh implementation of any one of the above aspects, the selector includes at least one of a router, a software switch, an extended Berkeley packet filter (eBPF) program, or a Maglev hash table, and the selector selects the at least one second node of the stateless forwarding layer for the extended endpoint.

[0017] In an eighth implementation of any of the above aspects, the extended endpoint provides at least one of a load balancer, a network address translator (NAT), a deep packet inspector (DPI), quality of service (QoS), a tunnel, a firewall, a protocol converter IPv6 / IPv4, an access control list (ACL), or a combination thereof.

[0018] In a ninth implementation manner of any one of the above aspects, the connection state tracking layer and the stateless forwarding layer are configured to scale independently of each other and not affect each other in the event of a failure.

[0019] In a tenth implementation manner of any of the above aspects, the DFT includes a Maglev hash table, and the Maglev hash table maps each first node chain of the DFT to at least one other first node chain of the DFT.

[0020] In an eleventh implementation manner of any of the above aspects, each node chain of the DFT includes an egress node that implements a co-located middlebox function, and the middlebox of the egress node implements all or part of the message processing function of the network service function described in the at least one specification table.

[0021] In a twelfth implementation of any of the above aspects, the at least one first node chain of the DFT implements a network service function for a second data packet, and the at least one second node of the stateless forwarding layer further receives the second data packet. Based on a determination that a second flow entry for the second data packet cannot be obtained from the DFT, the stateless forwarding layer selects a first node chain of the DFT using a Maglev hash algorithm, and sends the second data packet to a tail node of the selected first node chain of the DFT.

[0022] In a thirteenth implementation of any of the above aspects, the at least one second node of the stateless forwarding layer also receives a new flow table entry for the second data packet from the selected first node chain of the DFT, and sends the second data packet to a second destination based on the new flow table entry.

[0023] In a fourteenth implementation of any one of the above aspects, the tail node of the selected first node chain of the DFT retrieves an extended endpoint specification table, executes all middleboxes of the selected first node chain of the DFT according to the extended endpoint specification table, copies the extended endpoint specification table within an express data path (XDP) program of the tail node of the selected first node chain of the DFT, determines a new data flow table entry from processing the second data packet based on the extended endpoint specification table, and provides the determined new data flow table entry to the stateless forwarding layer.

[0024] In a fifteenth implementation of any of the above aspects, one or more processors implement a DFT operator that filters and predicts resource usage data collected by the DFT, determines multiple chains of express data path (XDP) programs to be added to or deleted from the DFT, recomputes a Maglev hash table that includes new chain assignments to track predecessor chains of each chain of the XDP program, and propagates the recomputed Maglev hash table to the first node of the DFT.

[0025] In a sixteenth implementation of any one of the above aspects, the header of the data packet includes a network service header, and the network service header includes metadata, which conveys the changes required for the DFT operation that the stateless forwarding layer will apply to the header of the received packet.

[0026] The apparatus can perform the method and can process instructions on a computer-readable medium, and other features of the method and instructions on the computer-readable medium are derived from the functionality of the apparatus. In addition, the explanations provided for each aspect and its implementation are equally applicable to the other aspects and corresponding implementations. Different embodiments can be implemented in hardware, software, or any combination thereof. In addition, any of the above examples can be combined with any one or more of the other examples to create new embodiments within the scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In the drawings, which are not necessarily drawn to scale, like numerals may describe like components in different views.The drawings illustrate generally, by way of example, and not by way of limitation, various embodiments described herein.

[0028] Figure 1 This shows that when adding a middlebox in a traditional system, connectivity is lost due to the lack of consistent state between the middleboxes.

[0029] Figure 2 The data plane worker node architecture in an exemplary embodiment is shown.

[0030] Figure 3 A droplet structure is shown in an exemplary embodiment.

[0031] Figure 4 The system architecture of the distributed flow table (DFT) layer and the stateless forwarding layer in an exemplary embodiment is shown.

[0032] Figure 5 The extended endpoint mapping to forwarders in an exemplary embodiment is shown.

[0033] Figure 6 It is shown that in an exemplary embodiment, a network service header (NSH) is used to convey message metadata.

[0034] Figure 7 The life cycle of a message in an extended endpoint implementing a secure load balancer use case in an exemplary embodiment is shown.

[0035] Figure 8 An extended endpoint programming model in an exemplary embodiment is shown.

[0036] Figure 9 The structure of NSH metadata with fields for the connection state tracking layer in an exemplary embodiment is shown.

[0037] Figure 10 A header structure with fields for a DFT operation in an exemplary embodiment is shown.

[0038] Figure 11 shows the general message flow in an exemplary embodiment of the system described herein.

[0039] Figure 12 A flow chart illustrating forwarder (FWD) behavior in an exemplary embodiment is shown.

[0040] Figure 13A 、 Figure 13B and Figure 13C Together, a flow chart illustrating the behavior of a flow table node (FTN) in an exemplary embodiment is shown.

[0041] Figure 14 A flow chart illustrating DFT automatic extension with chaining enhancement and Maglev hashing in an exemplary embodiment is shown.

[0042] Figure 15DFT scale-out (addition of FTN nodes) in an exemplary embodiment is shown.

[0043] Figure 16 The DFT horizontal shrinking (deletion of FTN nodes) in an exemplary embodiment is shown.

[0044] Figure 17 An FTN failure scenario is shown.

[0045] Figure 18 An active replication flow in an exemplary embodiment is shown.

[0046] Figure 19 The management plane architecture in an exemplary embodiment is shown.

[0047] Figure 20 An exemplary use case is shown showing east / west traffic in a data center where the destination address is used to extend the endpoint.

[0048] Figure 21 is an exemplary use case showing north / south forwarding traffic to a VM where the destination address is for another extension endpoint.

[0049] Figure 22 Shown Figure 21 In the example above, traffic flows in the north / south direction are reversed.

[0050] Figure 23A and Figure 23B An IPv4 to IPv6 transfer in an exemplary embodiment is shown.

[0051] Figure 24 is a block diagram of a circuit for executing the method provided by the exemplary embodiment. DETAILED DESCRIPTION

[0052] It should be understood at the outset that even though the following provides illustrative implementations of one or more embodiments, Figures 1 to 24 The disclosed systems and / or methods described can be implemented using any number of techniques, whether currently known or already in existence. The present invention is in no way limited to the illustrative implementations, drawings, and techniques shown below, including the exemplary designs and implementations shown and described herein, but may be modified within the scope of the appended claims along with their full scope of equivalents.

[0053] In one embodiment, the functions or algorithms described herein may be implemented in software. The software may include computer-executable instructions stored in a computer-readable medium or computer-readable storage device (e.g., one or more non-transitory memories or other types of hardware-based local or network storage devices). In addition, such functions correspond to modules, which may be software, hardware, firmware, or any combination thereof. Depending on the needs, multiple functions may be performed in one or more modules, and the described embodiments are merely examples. The software may be executed in a digital signal processor, an application specific integrated circuit (ASIC), a microprocessor, or other type of processor running in a computer system, such as a personal computer, server, or other computer system, thereby converting such a computer system into a specially programmed machine.

[0054] Figure 1 This shows that when adding middleboxes in a traditional system, connectivity is lost due to the lack of consistent state between the middleboxes. Figure 1 As shown, before the expansion, flows Flow-1 and Flow-2 reach destinations Dest-1 (110) and Dest-2 (120) through Middlebox-1 (100), while Flow-3 reaches destination Dest-3 (140) through Middlebox-2 (130). After the expansion to add Middlebox-3 (150) to handle Flow-2, Flow-2 is provided with an incorrect destination Dest-5 (160), and the connection must be re-established. Figure 1In

[15] , connection state is stored locally to each middlebox. When scaling out, flows are redistributed evenly across all middleboxes. Flow-2 is now mapped to the newly assigned Middlebox-3 (150), which unknowingly decides to send Flow-2 to Dest-5 (160) when it should have been sent to Dest-2 (120). This happens because consistent connection state is not maintained across multiple middleboxes during the addition of additional middleboxes. Due to the lack of a connection state tracking layer, when a new node / middlebox appears during scale-out, the new node / middlebox does not have the current flow state. Typically, during failover or scale-out / scale-in, connectivity loss or worse can lead to inconsistent packet forwarding decisions. Furthermore, the traditional approach of deploying network middleboxes in virtual machines results in underutilized resources and therefore inefficiencies due to a lack of resource sharing and differing network capabilities across multiple tenants. The following description describes systems and methods for addressing these and other issues in the design of a modular network service system. For example, in the described system, when a new forwarder is added, it will query the connection state tracking layer to know that Flow-2 will be sent to Dest-3 (140), thereby maintaining correct forwarding decisions.

[0055] Overall architecture

[0056] The modular network services system described in this paper is a two-layer system consisting of a forwarding layer and a link state tracking layer, which implements a distributed flow table (DFT) as an in-network key-value store. The forwarding layer is stateless, horizontally scalable, and failure-resistant independent of the link state tracking layer. The link state tracking layer's DFT maintains various types of state and is organized into chains of nodes, where each node in the chain replicates state and flow table entries using a unique, customized version of the chain replication protocol. Multiple chains can be automatically added or removed from the system using the chain enhancement process described in this paper. Network services are implemented as extended endpoints spanning multiple forwarder nodes and automatically scale based on traffic demand. Traffic to the forwarder nodes can be distributed using traditional protocols such as equal-cost multi-path (ECMP) routing and Open vSwitch (OVS) group selection, as well as a new selection protocol implemented in extended Berkeley packet filter (eBPF) programs.

[0057] The architecture described in this article decouples connection state tracking from middlebox decision making, allowing the two packet processing layers to scale independently and withstand failures. The connection state tracking layer is an in-network key-value store that implements multiple replicated DFT chains, each of which consists of multiple flowtable nodes (FTNs) that store connection state tracking data. Thus, a DFT is a linked list, a chain is a collection of FTNs, and an FTN contains the connection state. Together, these components comprise the connection state tracking layer.

[0058] Unlike typical key-value stores, DFT embeds its transactions within regular message flows, eliminating the need to establish connections to DFT nodes, buffer messages, acknowledge each message, or develop external protocols to support DFT operations. DFT maintains consistent connection state across replicas and scales horizontally based on the rate of new connections or connection closures. The forwarding layer, on the other hand, is a stateless forwarding layer that performs message forwarding decisions for already established connections and bounces messages for transmission. Individual nodes within the forwarding layer fail independently and scale horizontally quickly based on the total message rate of the network service. Together, these two layers form a scalable, resilient, and fault-tolerant distributed system of middleboxes that can replace existing hardware gateways.

[0059] The middlebox distribution system in an exemplary embodiment includes modular middleboxes with express datapath (XDP) programs that are chained together to form various types of network service functions. Natively sharing configuration across multiple XDP programs reduces the need for packets to traverse multiple nodes to make decisions about new flows. An overlay protocol integrated into the data plane provides consistent connection state, and built-in multi-tenancy enables infrastructure sharing for network services and middleboxes without the need to isolate middleboxes in virtual machines or containers, thereby reducing costs. In addition, native data plane failure detection minimizes the time to restore connectivity through the chat protocol.

[0060] Software Organization

[0061] Figure 2A data-plane worker node architecture 200 for running the modular network services system described herein is shown. Data-plane nodes (D-Nodes) 210 can be implemented as virtual machines (VMs), smart network interface cards (SmartNICs) supporting turnkey Linux distributions (e.g., PS225) or XDP offload, or other similar configurations in exemplary embodiments. The cluster in the exemplary embodiment manages two separate node pools. The first pool includes D-Nodes 210, which are responsible for all packet processing. The other pool includes management-plane nodes (M-Nodes) 220, which run a set of operators, which are custom software extensions of Kubernetes services. Custom resource operators implement management plane logic and manage D-Nodes 210 and middleboxes, as described below. D-Nodes 210 and M-Nodes 220 can dynamically join and leave the cluster, and D-Nodes 210 are not necessarily Kubernetes worker nodes and can be external to the Kubernetes cluster.

[0062] The M-Node 220 exposes network services as endpoints at a single IP address (VIP), and each M-Node 220 implements a chain of one or more middleboxes. Depending on bandwidth requirements, this network service is referred to herein as scaling endpoints (logical functions mapped to multiple forwarding nodes) across multiple D-Nodes 210, scaling endpoints horizontally, and scaling endpoints horizontally. The following is a non-exhaustive list of potential scaling endpoint types: load balancer, network address translator (NAT), deep packet inspector (DPI), quality of service (QoS), tunnel, firewall, IPv6 / IPv4 protocol converter, access control list (ACL), or a combination of these types. A typical example of a production system is an scaling endpoint that implements a secure load balancer and includes two middleboxes, each of which includes an ACL and a load balancer. Developers can specify and verify different scaling endpoint functions through a standardized interface.

[0063] Data plane node

[0064] Figure 2Each D-Node 210 of the type shown in can have multiple network interfaces. The management plane includes an M-Node 220 that interacts with multiple D-Nodes 210 and abstracts the network interfaces into droplets running on the D-Nodes 210. As used herein, a "droplet" provides a one-to-one mapping to a network interface. The droplet daemon (droplet-d) 221 includes a performance metrics library 222, an open source eBPF library 223 that provides an interface for user space to talk to the kernel, and a remote procedure call (RPC) interface 224 on the D-Node 210 that is exposed to the management plane so that the management plane can talk to and manage the D-Node 210. The droplet-d 221 implements the management plane by attaching an XDP program 230 to each interface (as described below with respect to Figure 3 Droplets are managed by the droplet-d 221 (described above) and populating eBPF maps (data structures shared between userspace programs and eBPF programs running in kernel space) 240, which store configuration, flow state, flow cache, and performance metrics. Droplet-d 221 also manages the data plane userspace middlebox 212, where packets can be processed in userspace 214 using address families from the Linux kernel 250 optimized for high-performance processing. AF_XDP sockets bring packets from the kernel 250 to userspace on the same node. Each middlebox 212 implements multiple XDP programs with local and shared data structures 216, which together reduce the need for packets to traverse multiple nodes to make decisions about new flows.

[0065] D-Node 210 also deploys a nomad driver 225 and an open source device plugin 226, which extend the Nomad agent / Consul 227 for health checks, telemetry, placement, autoscaling, and resource management. In the case of SmartNICs, the management plane components 221 to 227 of the D-Node can be deployed on a common server with multiple SmartNICs installed, and the server can act as an agent to manage the data plane of the D-Node 210 through RPC interfaces (including the RPC interface 224 of the droplet-d 221 and the RPC interface 218 of the D-Node 210).

[0066] Those skilled in the art will appreciate that, in addition to throughput and latency advantages, using SmartNIC as a D-Node 210 also has cost advantages over VMs. Furthermore, the cost of a typical 100 Gbps SmartNIC is only a fraction of the cost of a VM instance.

[0067] Droplet organization

[0068] Figure 3 The droplet structure in an exemplary embodiment is shown. Figure 3 yes Figure 2 A zoomed-in view of the architecture of the XDP program 230 is shown in Figure 1. Each droplet 300 runs an XDP program that implements data plane functionality. The droplet's XDP program includes a primary XDP (P-XDP) program 310, which receives input packets from a receiver RX, performs initial packet classification, and invokes middlebox functionality. In an exemplary embodiment, the middlebox is an eBPF program (M-XDP) that implements a ClickOS-like middlebox (e.g., middleboxes M-XDP-1, M-XDP-2, ..., M-XDP-N) 320 and related elements. The modular XDP programs M-XDP-1 through M-XDP-N 320 are dynamically loaded and unloaded and share an eBPF map 340 with the P-XDP 310. The P-XDP 310 calls the M-XDP program 320 via a communication medium (telecommunication channel and / or the Internet), and the M-XDP program 320 uses the AF_XDP interface to send data packets to the middlebox 330 in the user space. Figure 2 The user-space middlebox 212 in the data plane of the D-Node 210. The most important elements for most network services are IPRewriter, IPClassifier, and IPFilter. Two new M-XDP program 320 types are provided, which are specific to cluster functions: forwarder (FWD) and flow table node (FTN). Their functions are described in detail below.

[0069] The M-XDP program 320 runs entirely within the kernel 250 or is offloaded to a SmartNIC. For some complex middleboxes, the M-XDP program 320 can send packets to the userspace middlebox 330 via the AF_XDP interface. Both the P-XDP 310 and the M-XDP program 320 share the same eBPF map 340, an open-source data structure for XDP programs running within the kernel 250. The management plane provides packet classification rules, which the P-XDP 310 uses to determine which M-XDP program 320 should process the packet, tail-call the corresponding eBPF program, and provide the output packet at the message transmitter TX. A tail call is a function call within the eBPF infrastructure within the kernel 250 that enables code to communicate with each other without returning control to the calling program.

[0070] The droplet also maintains a set of eBPF maps 340 that are shared with all P-XDP 310 and M-XDP 320 programs. In the exemplary embodiment, the shared eBPF maps 340 have four categories:

[0071] Flow map (f-map) 342: A read / write map used by the middlebox to read / write flow state. A typical f-map 342 tracks the flow's internet protocol (IP) rewrite rules and connection state to the flow table entries, for example:

[0072] (saddr, sport, daddr, dport, protocol, vni) → (saddr', sport', daddr', dport', protocol', vni', conntrack),

[0073] Here, saddr is the source IP address, daddr is the destination IP address, port is the source port, dport is the destination port, protocol is the protocol number, and vni is the virtual network identifier. The ' symbol indicates a modified value for a field in the packet, and conntrack is a flag for tracking connection status (e.g., closed). Several types of f-map342 can be described for specific use cases.

[0074] Configuration map (c-map) 344: A read-only map that stores the configuration of different network services and middlebox configurations. Only the droplet-d 221 writes to c-map 344, while P-XDP 310 and M-XDP program 320 only read configuration from c-map 344. Several types of c-maps are described below.

[0075] Jump table (j-map) 346: The jump table 346 maps the M-XDP program 320 index to its eBPF file descriptor (index → ​​fd), allowing the P-XDP 310 to call the M-XDP program 320 and the M-XDP programs 320 to call each other.

[0076] Metering map (m-map) 348: The middlebox provides read / write mapping of packets and health indicators.

[0077] Management plane node

[0078] Users, administrators, and management plane operators interact with the node cluster in the exemplary embodiment through a Kubernetes application programming interface (API) server of the type described by Brewer in "Kubernetes and the path to cloud native," Proceedings of the Sixth ACM Symposium on Cloud Computing, pp. 167–167, 2015 (e.g., Figure 19 ) and described in more detail below. Through custom resource definitions, the API server functionality is extended to manage new types of objects specific to the system described here. Each object has a custom resource operator that manages its lifecycle from creation, issuance, to deletion through various workflows. Administrators deploy the operator as a pod integrated into the Kubernetes control loop.

[0079] The M-Node 220 uses Nomad (see nomadproject.io) and Consul (consul.io) clusters 227 to provide out-of-the-box services for large-scale clusters of thousands of D-Nodes 210. Similar to an operator, an administrator deploys the Nomad / Consul cluster 227 as a Kubernetes pod. Nomad exposes functionality to operators for placement and auto-scaling of various components. Consul provides automatic discovery of D-Nodes 210, where D-Nodes 210 can dynamically join or leave the data plane pool. Consul also provides a health check service via a chat protocol that enables the cluster to discover new bootstrap nodes. In addition, D-Nodes 210 can use Consul's chat protocol to quickly detect changes in the health status of other nodes, which is an important feature for fast failure recovery entirely within the data plane protocol. In addition, the health check service enables various operators to quickly activate workflows to replace failed nodes.

[0080] Data plane architecture

[0081] Figure 4 The overall system architecture in an exemplary embodiment is shown. The data plane consists of two layers: a distributed flow table (DFT) 400 and a stateless forwarding layer 410 including forwarders (FWDs) 412 (FWD-1), 414 (FWD-2), and 416 (FWD-N). FWDs 412-416 can be used to distribute data between multiple droplets 300 ( Figure 3) horizontally extends the eBPF program (M-XDP program 320). On the other hand, DFT 400 can be a group of FTNs (M-XDP type) organized as a chain. Figure 4 As shown, chain-C0 (420), chain-C1 (430), and chain-C2 (440) include FTNs 422-426 (chain-C0), FTNs 432-436 (chain-C1), and FTNs 442-446 (chain-C2), respectively, which replicate flow entries using a chain replication protocol of the type described in the following literature: Van Renesse et al., "Chain replication for supporting high throughput and availability," OSDI, volume 4, 2004; Jin et al., "Netchain: Scale-free sub-rtt coordination," 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18), pages 35–49, 2018; and Gaznavi et al., "Fault tolerance for service function chains," arXiv preprint arXiv:2001.03321, 2020. Figure 4 DFT 400 is shown with three chains 420, 430, and 440, each chain including three replicas and N FWDs 412-416 forming a stateless forwarding layer 410. The FTNs within a single chain can execute a custom chain replication protocol to maintain consistent flow state across the chain's replicas. Unlike existing fault recovery techniques, DFT 400 embeds the replication protocol header within the outer header of the packet, ensuring that packet transmission by any FTN or FWD is not delayed. In contrast, most existing fault recovery techniques buffer packets during the replication process, thereby delaying packet transmission. Furthermore, DFT 400 scales out / down the number of chains to match the connection rate in the cluster. In an exemplary embodiment, the Maglev consistent hashing algorithm (e.g., Eisenbud et al., “Maglev: A fast and reliable software network load balancer,” in 13th USENIX Symposium on Networked Systems Design and Implementation (NSD) 16, pp. 523–535, 2016) may be used to (re)partition flow entries across multiple chains 420, 430, and 440. In an exemplary embodiment, Figure 4 Each element identified in can be Figure 2 The architecture shown in is implemented as an XDP program.

[0082] DFT and FWD management

[0083] The FWD and DFT operators utilize the pool of D-Nodes 210 to place FWDs and FTNs as droplets 300. A droplet 300 can be either a FWD or an FTN, and FTNs and FWDs cannot coexist on the same D-Node 210. However, multiple FWDs can share the same D-Node 210, but each FWD is placed on a separate droplet 300. The droplet 300 that implements the FTN also loads all middleboxes supported by the cluster as M-XDP programs 320. Except for the M-XDP program 320 collocated with the tail FTN, all M-XDP programs 320 are idle. For example, in Figure 4 In the example, middleboxes 428, 438, and 448 collocated with FTN-T3, FTN-T6, and FTM-T9, respectively, are all active and include one or more logical functions. For a single DFT 400, the number of FTNs in the chain is static. A single chain has at least two FTNs. The DFT operator designates one FTN in the chain as the tail node, one FTN as the head node, and the remaining nodes (if any) as replica nodes. For example, Figure 4 Chain-C0 in FIG400 has FTN-T1 422 as the head node, FTN-T2 424 as the replica node, and FTN-T3 426 as the tail node. The number of chains can be increased and decreased for horizontal expansion and contraction. The DFT operator maintains a Maglev hash table 450 and propagates the table 450 to FWDs 412-416 or any other entity that can communicate with DFT 400 (e.g., host nodes, other operators).

[0084] Extended endpoint and forwarder selection

[0085] Extension endpoints span multiple FWDs. A FWD selector terminates flows assigned to an extension endpoint to one of the FWDs. The extension endpoint operator places, scales out, and scales out extension endpoints across multiple FWDs based on their bandwidth requirements. Extension endpoints belong to different tenants, and multiple extension endpoint types share the same FWD.

[0086] Figure 5 Four extended endpoints 540, 550, 560 and 570 are shown for messages entering the system, along with different types of FWD selectors. Figure 5The FWD selectors in the diagram are data center routers 510 (e.g., using equal-cost multi-path (ECMP) routing protocols and / or border gateway protocol (BGP)), software switches 520 on host VMs 522 (e.g., OVS using group selection hashing), and VMs 532 using simple hashing or eBPF programs 530 on the container's egress path. As shown, the extended endpoint operator encapsulates one tenant's north / south packets onto a set of FWDs at the forwarding layer 410 that also share another tenant's load balancer. This approach ensures that the system can be deployed with both traditional cloud networks and state-of-the-art host networking approaches. The extended endpoint operator keeps the data center routers or host switches updated by transforming the extended endpoint-to-FWD mapping.

[0087] Extended endpoints 540-570 are mapped to multiple FWD nodes of forwarding layer 410, and one or more of a number of techniques can be used to select one of the FWD nodes of forwarding layer 410 to which a packet is to be sent. In this example, the extended endpoint operator places three extended endpoints 540, 550, and 560 on the FWD of forwarding layer 410, where the three FWDs of forwarding layer 410 are shared across the three extended endpoints 540, 550, and 560. In this scenario, the extended endpoint operator also programs ECMP router 510 to select the FWD of extended endpoint 560, the open source switch OVS receives a packet from VM 522 running on a host and selects one of the FWD nodes of the second extended endpoint 550, and the eBPF program 530 on another host selects the third extended endpoint 560. For example, in the case of a simple hash, when a packet is sent from VM 532 to eBPF program 530, eBPF program 530 performs a simple hash on the packet and selects one of the FWD nodes of extended endpoint 560. Fourth extended endpoint 570 is detached. Here, the extended endpoint operator does not assign extended endpoint 570 to any FWD in forwarding layer 410. In practice, for detached extended endpoint 570, the host sends the flow directly to the FTN tail node (e.g., tail node 426 with active middlebox 428) of one of the chains (e.g., chain 420) selected using Maglev hash table 580. This FTN tail node is used when a SkilledEndpoint is mapped directly to an FTN node rather than an FWD node. However, in exemplary embodiments, Maglev hashing can also be used to assign packets to the FWD node of the extended endpoint.

[0088] In the case of a separate extended endpoint 570, the FWD functionality is built into the host. The benefits of a separate extended endpoint 570 are that it bypasses the forwarding layer 410 and provides the ability to develop a custom hash function for chain selection. The following discusses the separate extended endpoint scenario in conjunction with the east / west traffic use case.

[0089] The above architecture enables automatic expansion of DFT 400, which eliminates the operational overhead of adding new flow tracking nodes. In addition, DFT 400 provides in-network functionality to track all types of state changes in addition to flow tracking. Compared to the three-tier architecture, the two-tier system described in this article further minimizes the number of hops to establish a connection. In addition, the middlebox decision-making collocated with the tail node can make configuration changes take effect immediately on existing flows without the need for an external synchronization mechanism. As described below about Figure 14 As explained, the FWD of the forwarding layer 410 also provides a self-verification function without the need for external synchronization.

[0090] Modular network services

[0091] The system described herein also uses a declarative programming approach that transforms a high-level middlebox description into a specification sheet. The specification sheet defines the modular sequence of actions required to implement the middlebox functionality (network service). A mechanism is also provided to verify the correctness of the specification sheet. The system's modules are also "decoupled," with their execution split across different nodes in the forwarding layer 410. The specification sheet and custom message format enable the execution of the specification sheet even when the modules do not necessarily run on the same nodes. Modules in the system can also be added or removed without any service disruption or new deployments.

[0092] The system handles both east-west and north-south traffic. For east-west traffic, endpoint packets are tunneled using a popular tunneling protocol (typically VxLAN or Geneve). For north-south traffic, the gateway extends the endpoint to receive external traffic from the service provider, where the FWD tunnels the packets. Within DFT 400 and between FWDs 412-416, the network service header (NSH) is used to convey packet metadata.

[0093] Figure 6 In an exemplary embodiment, NSH 600 is shown using a VXLAN packet format to transmit application-dependent packet metadata (e.g., metadata type-2 610), a service path identifier (SPI) / service index (SI) field 620, an NSH basic header 630, or any suitable combination thereof. Figure 6As shown, a VXLAN packet includes an outer Ethernet header 640, an outer IP header 645, an outer UDP header 650, and a VXLAN / Geneve header 655, which provide VXLAN encapsulation of the original packet, which includes an inner Ethernet header 660, an inner IP header 665, an inner UDP header 670, and a payload 675. For more information on NSH, see Paul Finn, Uri Elzur, and Carlos Pignataro, "Network Service Header (NSH)", RFC 8300, RFC Editor, 2018. Substrate Networks is not expected to perform any routing decisions based on NSH 600 to achieve system functionality. The content of metadata type-2 610 within NSH 600 has a certain size, which depends on the application, as will be referenced below. Figure 9 and Figure 10 Discuss in more detail.

[0094] Extended endpoint modular programming

[0095] The SPI / SI field 620 of the NSH identifies the extended endpoint. The forwarding layer 410's forwarding device (FWD) maps packets to SPIs. The extended endpoint operator populates a configuration map (SPI-c-map) entry in each forwarding layer 410 FWD where the extended endpoint is located. The packet-to-SPI mapping in the SPI-c-map is typically (vni, daddr, dport) → spi. The NSH's SI in the SPI / SI field 620 identifies the middlebox of the extended endpoint. The P-XDP 310 and M-XDP program 320 use the SPI and SI values ​​to determine the next middlebox that should process the packet. When a packet is transferred from one middlebox to the next, the SI value is decremented by one or more values ​​according to the extended endpoint specification table. The extended endpoint operator programs its specification table in each FWD of the extended endpoint and in all FTNs of the DFT 400. The configuration map (extended endpoint-c-map) stores the mapping of SPIs to its specification table.

[0096] For example, consider implementing a load-balancer (LB) extension endpoint behind an access control list (ACL). Figure 7 The life cycle of a message in an extended endpoint implementing a secure load balancer use case in an exemplary embodiment is shown. Figure 7Droplet-2 700 is shown communicating with Droplet-1 710 and head node 706. Detailed SI values ​​used for message processing are shown in specification table 720. Droplet-2 700 includes P-XDP classifier 702, LB M-XDP 704, and monitoring (MON) M-XDP 708. Droplet-1 710 includes P-XDP classifier 712, ACL M-XDP 714, and FWD 716. Droplet-1 710 is shown twice to illustrate that different parts of Droplet-1 710 are used at different points in communication with Droplet-2 700.

[0097] When a message first arrives from the source (SRC) at Droplet-1 710 at one of the FWD nodes of the extended endpoint, the P-XDP classifier 712 looks up the SPI of the extended endpoint, inserts the NSH 600, sets the SPI value of the header, and sets the SI to 255. The message then proceeds as follows (e.g. Figure 7 The SI values ​​are shown in Table 720):

[0098] SI:255 Action: P-XDP classifier 712 of droplet-1 710 tail calls ACL M-XDP 714:

[0099] ACL M-XDP 714 performs stateless packet filtering and can discard packets

[0100] ACL M-XDP 714 reduces SI by one

[0101] SI:254 Action: ACL M-XDP 714 tail calls M-XDP FWD 716 of droplet-1 710:

[0102] The FWD 716 searches its flow cache to find an existing entry for the flow of the packet.

[0103] If the flow entry exists, the FWD 716 reduces the SI to 247, otherwise it reduces the SI to 253.

[0104] SI:253 Action: FWD 716 sends message to droplet-2 700

[0105] The P-XDP classifier 702 of Droplet-2 700 searches its flow cache for an existing entry for the packet's flow.

[0106] If the flow entry exists, the P-XDP classifier 702 reduces the SI to 252, otherwise it reduces the SI to 251.

[0107] SI:252 Action: P-XDP classifier 702 of droplet-2 700 sends the message to its original source (droplet-1) 710

[0108] P-XDP classifier 702 reduces SI by one

[0109] SI:251 Action: The P-XDP classifier 702 of Droplet-2 700 tail calls the message of LB M-XDP 704:

[0110] LB M-XDP 704 determines the IP address of the backend to which the message is to be sent

[0111] LB M-XDP 704 reduces SI by one

[0112] SI:250 Action: LB M-XDP 704 sends the message to the head node 706 for replication

[0113] The head node 706 updates the flow table entry according to the replication protocol discussed below

[0114] The last DFT copy decrements SI by one

[0115] SI:249 Action: SI is also used to process messages during replication. For the sake of brevity, the details of the protocol are omitted here as they will be described below.

[0116] SI:248 Action: The last DFT replica sends the message to MON M-XDP 708 of Droplet-2 700.

[0117] MON M-XDP 708 Update m-map.

[0118] MON M-XDP 708 reduces SI by one

[0119] SI:247 Action: MON M-XDP 708 sends the message to its original source (Droplet-1 710).

[0120] The P-XDP classifier 712 of droplet-1 710 decrements the SI by one.

[0121] SI:246 Action: Droplet-1 710's P-XDP classifier 712 tail calls Droplet-1's M-XDP FWD 716

[0122] The FWD 716 of Droplet-1 710 caches the flow entry and sends the packet to its destination (DST).

[0123] Specifications

[0124] In an exemplary embodiment, the extended endpoint specification form enables flexible programming of extended endpoint logic in a modular fashion. Most specification branches will invoke the M-XDP program 320 on the same droplet 300, with the exception of cases where flow state must be updated via the DFT 400. In both cases, the specification itself is sufficiently general that the middlebox operator can be placed anywhere in the node cluster. This programming model enables extended endpoint administrators to design extended endpoint logic using a high-level description language implemented by management nodes (e.g., nodes in the management plane).

[0125] Figure 8 An extended endpoint programming model 800 in an exemplary embodiment is shown. The extended endpoint programming model 800 includes an extended endpoint configuration 810 (provided in an extended endpoint description language), which is processed by an extended endpoint operator 820 to generate a specification table 830. A functional verifier 840 takes as input a middlebox specification 850 and the specification table 830 and generates a proof or counter example 860 to prove whether the specification table 830 is valid.

[0126] The extended endpoint operator 820 is about Figure 5 The extended endpoint configuration 810 specifies the function to be executed by the extended endpoint, written in a high-level description language. The high-level description language is converted into a specification table 830 in real time, specifying the series of middlebox actions required to implement the extended endpoint's function. The specification table 830 is sent to all nodes so that they know how to process the message upon receiving it. A function verifier 840, such as Gravel (Zhang et al., "Automated Verification of Customizable Middlebox Properties with Gravel," 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20), pp. 221–239, 2020), takes the specification table 830 and the predetermined specifications of the middlebox 850 as input and determines whether the extended endpoint operator 820 has correctly transformed the user-specified function into what the system should do. The verifier 840 provides a proof or counter example 860 of the correctness of the extended endpoint configuration. If proven to be correct, the extension endpoint propagates the specification table 830 to the FWD of the extension endpoint in the forwarding layer 410 and the DFT 400. For example, Figure 7In the example shown in Figure 7, the extended endpoint propagates specification table 720 to the node to implement a load balancer behind the access control list. Therefore, the node will include the same specification table 830, but different users can access different configuration tables in different middleboxes based on the received message, and the corresponding extended endpoint will execute the actions specified in specification table 830. Spec table 830 will be pushed to idle and active middleboxes. When receiving a message, the FTN node will classify the message as belonging to a certain extended endpoint, and will retrieve the corresponding specification table 830 for the extended endpoint and execute the actions listed in specification table 830.

[0127] Therefore, unlike traditional approaches, this system can implement several types of "distributed middleboxes" without requiring code changes and separate deployment. The system also provides module verification capabilities and a declarative programming interface that is aware of the distributed nature of the system. Modules can be independently extended, can be reused by several middlebox types, and do not need to be deployed on the same machine.

[0128] DFT internal protocol

[0129] In an exemplary embodiment, if a middlebox decision has been made for a flow, the FWD node of forwarding layer 410 immediately modifies the packet and forwards it to its destination. If the FWD node is unaware of the decision, such as in the case of a failure, horizontal expansion, or a new flow, the packet is sent to the tail node of one of the DFT chains (e.g., chain 420, chain 430, or chain 440). The chain is selected using a Maglev hash. If the tail node finds the flow entry, it replies to the FWD with the pre-made decision including the flow entry. If the flow entry does not exist, the tail node makes a new decision to create a new flow entry based on the specification table and replicates the decision in the chain.

[0130] Figure 9 The structure of NSH MD Type 2 metadata 900 with fields for DFT 400 in an exemplary embodiment is shown. NSH MD Type 2 metadata 900 is primarily designed around the DFT operation to communicate the required changes that FWD must apply to the header of a message. The fields include:

[0131] Version (3 digits) 910: Metadata version

[0132] Length (5 bits) 920: Total length of metadata in 4-byte increments

[0133] Reverse (R) (1 bit) 930: Indicates DFT operation for reverse flow

[0134] Reserved (23 bits) 940: For future use and must be zero

[0135] FWD IP address (32 bits) 950: IP address of the FWD to which the message is returned

[0136] DFT Operation 960: Variable Length Table Operation

[0137] The system's NSH MD Type 2 metadata 900 can package multiple DFT operations in the same metadata header as long as there is space in the message allowed by the network's maximum transmission unit (MTU). Figure 10 The header structure of the DFT operation 1000 is shown with fields in an exemplary embodiment. The fields include:

[0138] Op (4 bits) 1010: Metadata version

[0139] –0x0: Reserved

[0140] –0x1: Lookup

[0141] –0x2: Reply

[0142] –0x3: Update ANY

[0143] –0x4: Update NOEXIST

[0144] –0x5: Update EXIST

[0145] –0x6: Delete

[0146] –0x7-0xf: Reserved

[0147] Reserved (2 bits) 1020

[0148] Index (16 bits) 1030: Index of the eBPF map for FTN

[0149] Key Len (5 bits) 1040: Key length, multiple of 4 bytes

[0150] Val Len (5 bits) 1050: value length, multiple of 4 bytes

[0151] Key 1060: Length of the key payload (Key Len)

[0152] Value 1070: Value payload of value (Val Len)

[0153] The maximum length of the key and value fields fits within the IP + Transmission Control Protocol (TCP) header. The exact structure of the key and value fields is preserved because they are related to the key and value structure of the eBPF map referenced by the SI of the SPI / SI field 620 in the NSH header 600. Typically, the optimized key is used as a reference field in internal message headers to save space in metadata headers and avoid duplicating unnecessary information that already exists in the message.

[0154] Unlike traditional systems, the use of a unique and customized NSH (Metadata Type-2) 600 reduces the overlay overhead required for DFT operations and intercommunication between distributed modules across nodes. The protocol described herein enables the transfer of general status information to DFT 400 nodes for flow tracking without requiring packet buffering or additional acknowledgment mechanisms. Decisions regarding new flows are made on the same tail FTN node, without requiring additional hops to an additional decision-making layer. Furthermore, the approach described herein enables rapid scale-out / scale-out, automatic synchronization during failure recovery, and immediate configuration propagation, without the need for external synchronization or a separate real-time synchronization protocol. Active replication or synchronization can still be used to supplement this passive replication protocol as an optional optimization.

[0155] Message Flow

[0156] Figure 11 shows the general message flow in an exemplary embodiment of the system described herein. Figure 11 A typical example of the interaction between the FWD of the forwarding layer 410 and the DFT 400 is described. Algorithm 1 ( Figure 12 ) and Algorithm 2 ( Figure 13A and Figure 13B ) details the FWD and FTN behaviors used for message processing. Figure 11 As shown, the message goes from source to destination through steps 1-7 as follows:

[0157] 1. FWD-1 1100 receives a message from source 1110.

[0158] 2. FWD-1 1100 searches its flow f-map (flow cache) for the flow entry for the packet.

[0159] If an entry exists, FWD-1 1100 rewrites the packet's header according to the flow entry and bounces the packet for transmission to its destination 1120.

[0160] b. If the table entry does not exist, FWD-1 1100 performs a hash operation on the tuple of the message and selects the corresponding chain from the Maglev table, adds an NSH header corresponding to the extended endpoint of the message (using the lookup opcode), and then sends the message to the tail FTN of the chain ( Figure 11 If the message is not encapsulated (external traffic), FWD-1 1100 encapsulates the message.

[0161] 3. When the tail FTN-T3 1130 receives the message (at 2(b)), it looks up the message flow entry in its flow f-map. There are two possible scenarios:

[0162] a. If the entry exists, FTN-T3 1130 replies with the new flow entry and sends it to FWD-1 1100 (using the Reply opcode).

[0163] b. If no entry exists, it is assumed to be a new flow and FWD-1 1100 tail calls the first M-XDP in the extended endpoint specification table. The specification table continues until the last flow entry ends, then M-XDP adds the update DFT operation to NSH 600 and sends the message to the head of the chain FTN ( Figure 11 FTN-T1 1140 in.

[0164] 4. The fourth message can be:

[0165] a. When FWD-1 1100 receives the message (at 3(a)), it caches the flow entry, rewrites the message header according to the new flow entry, and bounces the message for transmission to the destination 1120.

[0166] b. When the head FTN 1140 receives the message (at 3(b)), there are two possible scenarios before the head FTN 1140 sends the message (at 4(b)):

[0167] 5. When the replica FTN-T2 1150 receives message 4(b), it updates the flow entry according to the update request in message 4(b) - regardless of whether the flow entry is expired or does not exist.

[0168] 6. When tail FTN-T3 1130 receives packet 5, it updates the flow entry—regardless of whether the flow entry is expired or does not exist. Tail FTN-T3 1130 then sends the packet with the new flow entry to FWD-1 1100. However, if the reverse flow hashes to a different chain, FTN-T3 1130 updates the reverse flow entry with a similar packet flow (starting from 3(a)).

[0169] 7. As in 4(a), when FWD-1 1100 receives packet 6, it caches the new flow entry options, rewrites the packet’s header according to the new flow entry, and bounces the packet for transmission to destination 1120.

[0170] Figure 12 The example embodiment shows a repeater (eg, Figure 11 The following Algorithm 1 flowchart 1200 illustrates the behavior of FWD-1 (1100) in the forwarding node. As shown, the forwarding node encapsulates packets without a flow or state entry in the forwarding node's FWD table into the DFT tail. A custom bearer protocol (NSH) is used to define all connection state tracking transactions (Lookup, Reply, Update Any, Update No-Exist, Update Exist, and Delete). The forwarding node selects the flow chain using consistent hashing. No packet buffering, retry attempts, or acknowledgment (ACK) messages are provided at the forwarder layer or between forwarding nodes and flow table nodes.

[0171] like Figure 12 As shown, at operation 1210, the FWD receives a packet from the source (which can be packet 1, 3a, or 6). At operation 1220, the FWD searches the f-map in its flow cache for the flow entry for the packet. If, at operation 1230, it is determined that a flow entry exists, then at operation 1240, the packet is modified and sent to destination 1120 (packet 2(a)). If, at operation 1230, it is determined that no flow entry exists, then at operation 1250, it is determined whether the packet contains an NSH header and whether the operation code is "Reply." If so, at operation 1260, the FWD's flow cache is updated, the NSH header is deleted, and the packet is modified and sent to destination 1120 (packet 4(a) or 7). On the other hand, if, at operation 1250, it is determined that the packet does not contain an NSH header and the operation code is "Reply," then at operation 1270, it is determined whether the packet is encapsulated. If not, then at operation 1280, the packet is encapsulated. Otherwise, at operation 1290, the NSH 600 is added to the message, the FWD IP address is pushed to the header, the tail address is obtained using Maglev hashing, and the message is sent to the tail node of the selected chain (Message 2(b)).

[0172] about Figure 12 The described FWD behavior can be expressed in Algorithm 1 as follows:

[0173] Algorithm 1: FWD behavior

[0174] Data: p: packet, nsh: NSH header, ftn ip, ip: FWD IP address. f : flowentry, chain: DFT chain of the flow

[0175] 1FWD receives p from source;

[0176] 2f := LookupFlowCache(p);

[0177] 3 iff existsthen

[0178] 4ModifyPacket(p, f );

[0179] 5returnTransmit p to destination;

[0180] 6 ifp contains an NSH header and n.op is Replythen

[0181] 7f := p.nsh.value ;

[0182] 8UpdateFlowCache(p, f );

[0183] 9RemoveNshHeader(p);

[0184] 10ModifyPacket(p, f );

[0185] 11returnTransmit p to destination;

[0186] 12f := LookupFlowCache(p);

[0187] 13 ifp is not encapsulatedthenEncapsulate(p); ;

[0188] 14AddNshHeader(p) ;

[0189] 15p.nsh.fwd ip := ip ;

[0190] 16p.nsh.Op := Lookup ;

[0191] 17chain := MaglevHashingForwardFlowChain(p);

[0192] 18 returnTransmit p to chain.tail;

[0193] Figure 13A 、 Figure 13B and Figure 13C Together, the following flowchart of Algorithm 2 illustrates the behavior of the FTN in a DFT chain in an exemplary embodiment.

[0194] Figure 13A The example embodiment shows the tail node (eg, Figure 11 The behavior of the FTN-T3 1130) is shown in the following Algorithm 2 flowchart 1300. Figure 13A As shown, during a flow lookup (eg, performed by FWD-1 node 1100), at operation 1302, FTN-T3 1130 receives a message with an opcode lookup ( Figure 11 Message 2(b) in ). If it is determined at operation 1304 that a flow entry exists, then at operation 1306, the tail FTN-T3 1130 updates the NSH key / value and sends the message ( Figure 11 3(a) in the flow table) is sent to the FWD-1 node 1100. If it is determined at operation 1304 that there is no flow table entry, the FWD-1 node 1100 determines the output flow of the given specification table by activating multiple XDP programs that implement the middlebox function. Specifically, at operation 1308, the tail FTN-T3 1130 retrieves the information about Figure 8 The created extended endpoint specification table is then executed by all middleboxes in the chain according to the specification table. Then, at operation 1310, the tail FTN-T3 1130 commits the newly decided flow entry by forwarding a message with the opcode UPDATE NO_EXIST as a key / value entry to the head FTN 1140. The head FTN node 1140 is typically the head node of the current or predecessor chain. If horizontal expansion or contraction is in progress, the head FTN node 1140 is the head of the processor chain for that flow. FWD-1 1100 is aware of existing flow entries and reverse flow entries within a packet hop. FWD-1 1100 only contacts the tail FTN-T3 1130 in the DFT 400. When the tail FTN-T3 1130 initiates a flow update, it maintains the first-in, first-out (FIFO) link property required for chain replication correctness.

[0195] Figure 13B The head node (eg, Figure 11 The behavior of the FTN-T1 head node 1140) is shown in the following Algorithm 2 flowchart 1320. Figure 13BAs shown, at operation 1322, the head FTN node 1140 receives a message with the operation code UPDATE NO_EXIST (message 3(b)). At operation 1324, the head FTN node 1140 updates the flow entry only when the flow entry does not exist or has expired, and updates the reverse flow entry if the chain is also a reverse flow chain. At operation 1326, if the flow entry exists, the head FTN node 1140 modifies the operation code to Update ANY and sends the message to the next node in the chain (message 4(b)). Therefore, the head FTN node 1140 indicates the recovery of an interrupted replication flow, the recovery of a failed tail node, or repartitioning during chain enhancement (horizontal expansion / horizontal contraction). Passive repartitioning can be performed during horizontal expansion / horizontal contraction of the DFT chain. Passive synchronization of flow entries can also be performed for a new FTN node that replaces a failed FTN node.

[0196] Figure 13C The following flowchart 1330 of Algorithm 2 is shown, illustrating the behavior of an FTN node in DFT 400 that receives a message with an operation code of UPDATEANY, EXIST, or DELETE. Figure 13C As shown, at operation 1332, the FTN node receives a message with an operation code of UPDATE ANY, EXIST, or DELETE ( Figure 11 4 (b) or 5), and updates or deletes the forward and reverse flow table entries at operation 1334 and resets their expiration timers. At operation 1334, if the chain is also a reverse flow chain, the FTN node also updates or deletes the reverse flow table entries. If it is determined at operation 1336 that the FTN is the tail of the chain, then at operation 1338 the tail FTN uses the message ( Figure 11 Otherwise, at operation 1340, the FTN sends the message as is to the next FTN node in the chain (e.g., Figure 11 Message 5).

[0197] about 13A to 13C The described FTN node behavior can be expressed in Algorithm 2 as follows:

[0198] Algorithm 2: FTN behavior

[0199] Data: p: packet, nsh: NSH header

[0200] Data:f : flow entry, k := (saddr, sport, daddr, dport, protocol,vni): flow entry key

[0201] 1chain := MaglevHashingForwardFlowChain(p) ;

[0202] 2chain ′ := MaglevHashingReverseFlowChain(p) ;

[0203] 3 ifp.nsh.Op is Lookupthen

[0204] 4f := LookupFlowEntry(p);

[0205] 5iff existsthen

[0206] 6p.nsh.Op := Reply ;

[0207] 7p.nsh.value := f ;

[0208] 8returnTransmit p to p.nsh.fwd ip ;

[0209] 9f←Get New Flow Entry For Scaled Endpoint ;

[0210] 10p.nsh.Op := Update NOEXIST ;

[0211] 11p.nsh.key := k ;

[0212] 12p.nsh.value := f ;

[0213] 13returnTransmit p to chain.head ;

[0214] 14ifp.nsh.Op is Update NOEXISTthen

[0215] 15UpdateForwardFlowEntryIfNotExist(p.nsh.index, p.nsh.key,p.nsh.value);

[0216] 16ifchain equals chain ′ then

[0217] 17UpdateReverseFlowEntryIfNotExist(p.nsh.index, p.nsh.key,p.nsh.value);

[0218] 18ifFTN is chain.headthen

[0219] 19f := LookupFlowEntry(p);

[0220] 20p.nsh.value := f ;

[0221] 21p.nsh.Op := Update ANY;

[0222] 22 ifp.nsh.Op is Update ANY or Update EXIST or Deletethen

[0223] 23UpdateOrDeleteForwardFlowEntry(p.nsh.index, p.nsh.key,p.nsh.value);

[0224] 24ifchain equals chain ′ then

[0225] 25UpdateOrDeleteReverseFlowEntry(p.nsh.index, p.nsh.key,p.nsh.value);

[0226] 26ResetForwardFlowEntryExpiryForUpdate(p.nsh.index, p.nsh.key,p.nsh.value);

[0227] 27 ifchain equals chain ′ then

[0228] 28ResetReverseFlowEntryExpiryForUpdate(p.nsh.index, p.nsh.key,p.nsh.value);

[0229] 29 ifFTN is chain.tailthen

[0230] 30ifchain NOT equals chain ′and p.nsh.r bit is NOT setthen

[0231] 31Set(p.nsh.r) ;

[0232] 32returnTransmit p to chain ′ .head;

[0233] 33returnTransmit p to p.nsh.fwd ip ;

[0234] 34 returnTransmit p to next FTN in chain.

[0235] DFT expansion

[0236] Figure 14 A flowchart 1400 illustrates automatic expansion of a DFT 400 with chain augmentation and Maglev hashing in an exemplary embodiment. As shown, at operation 1410, the DFT operator (executing in the management plane) continuously monitors resource usage at the tail FTN node of each chain. Resources typically, but not necessarily, include bandwidth and memory utilization. At operation 1420, the DFT operator filters and predicts the collected load metrics for each chain and determines the number of chains to add or remove from the DFT 400. At operation 1430, the DFT operator recalculates the Maglev hash table 1435 to include new chain assignments for a successor chain, new chain C1, or recalculates the Maglev hash table 1440 when chain C1 is removed. For example, Maglev hash table 1435 begins with the hashes of chains C0 and C2 and is modified to include the hash of chain C1 when chain C1 is added during horizontal expansion. Similarly, the Maglev hash table 1440 starts with the hashes of C0, C1, and C2, and is modified to delete the hash of chain C1 during a horizontal contraction. For each horizontal expansion or horizontal contraction action, a chain may be added or deleted. At operation 1445, after horizontal expansion or horizontal contraction to the FTN node of the predecessor chain, the process propagates a new Maglev hash table 1435 or 1440 for each chain. Using the Maglev hash table to maintain the mapping between the chain and its predecessor chain in this way enables the system to perform horizontal expansion and horizontal contraction operations without external synchronization. The following will refer to Figure 15 and Figure 16 Describe examples of horizontal expansion and horizontal contraction.

[0237] As described above, the scale-out / scale-out process is the process of adding additional chains to the DFT or removing existing chains from the DFT. This is done to optimize the flow decision load by redistributing flow table entries across the chains and their FTNs. By relying on two replication protocols and a process called "chain strengthening" described below, we can ensure an even distribution of all flow table entries while also preserving previously made flow decisions.

[0238] Chain Enhancement

[0239] When chains are deleted (horizontally contracted) or added to the DFT 400 (horizontally expanded), the management plane's M-Node 220 performs chain reinforcement. In either scenario, the M-Node 220 connects the chains by first identifying the "predecessor" and "successor" chains. Next, the tail node of the predecessor chain is identified as the pseudo-head of the successor chain. Finally, both active and passive replication work together to repartition and redistribute flow entries across the active chains.

[0240] As an alternative to adding a chain at the head, adding a predecessor chain at the tail of the successor chain will cause additional load on the tail of the predecessor chain. Figure 15 The following scenario in the example shows DFT horizontal expansion (adding FTN nodes), where FTN-T3 426 is still active and the newly created chain-C1 is also enhanced. In this case, FTN-T3 426 will be used to hash to the flow of chain-C0 and chain-C1.

[0241] Rezoning

[0242] There are two methods for repartitioning: passive and active. In both cases, the M-Node 220 performs chain enhancement according to the replication protocol. Figure 15 In the example, FTN-T9 446 (which is the tail of chain-C2 440) and FTN-T3 426 (which is the tail of chain-C0 420) become pseudo-headers of chain-C0 420 and chain-C1 430, respectively. The active replication protocol is activated, using the active middlebox of FTN-T6 436 to replicate the new chain index from its predecessor to its successor chain to decide how to process packets. All flows from indexes 0 and 2 of FTN-T3 426 are hashed to FTN-T4 432 of chain-C1 430, thereby being replicated across chain-C1 430. Simultaneously, given the previously described packet flows, the passive replication protocol preserves and replicates the decisions previously made in FTN-T9 446 and FTN-T3 426.

[0243] For example, suppose from Figure 15FTN-T6 436 of chain-C1 430 receives a message from the forwarding layer 410's FWD with a Lookup opcode for a flow that hashes to table index 2 of table 1500. FTN-T6 436 then sends an Update Flow request (message 3(b)) to the pseudo-header 1510 (FTN-T3) of chain-C1, which also serves as the tail of chain-C0 420. If FTN-T3 1510 finds an entry for the flow that now belongs to chain-C1 430, FTN-T3 1510 ignores the Update Flow request and copies the existing entry to FTN-T4 432 (message 4(a)). Ultimately, any flow entry that hashes to table index 2 is copied across chain-C1 430 at messages 5 and 6 and deleted from the node of chain-C0 430. Through a similar process, FTN-T9 446 can become the pseudo-head 1520 of chain-C0'.

[0244] Chain enhancement can prevent race conditions between the active replication protocol and the passive replication protocol when changing FTN flow entries. Since active replication only takes a snapshot of FTN flow entries, chain enhancement is relied upon to prevent existing flow entries from being overwritten by passive replication.

[0245] For example, consider Figure 16 The following is a DFT horizontal contraction (FTN node deletion) scenario, in which the tail node FTN-T6 436 of chain C1 is deleted. Without chain enhancement, a snapshot of FTN-T5's flow table entries is taken from table 1600, and the flow table entries are propagated to all FTNs in chain C0, starting with FTN-T1 422 (messages 4(b), 5, and 6). With passive replication occurring simultaneously, assume that tail node FTN-T3 426 writes a new key-value pair flow table entry to FTN-T1 422. This may cause the flow entry to fail to be retained, as FTN-T1 422 will now ignore any actively replicated writes using the same key. With chain enhancement, FTN-T3 426 will attempt to write to FTN-T5 1610 (message 3(b)). FTN-T5 1610 will retain its own flow table entry by ignoring any matching new writes and replicating its existing entries. The deleted FTN-T4 432 in the chain C1 may also become the pseudo-header FTN-T4 1620 .

[0246] Adding a new chain

[0247] In the case of adding a large number of extension endpoints to a node cluster, the flow table entries may overload the system memory of all FTNs in a single chain. For such scenarios, new chains may be issued to handle the new extension endpoints. The assignment of extension endpoints to chains will be determined by the number of extension endpoints in a given chain, where the chain with the least number of extension endpoints will have priority.

[0248] FTN troubleshooting

[0249] In the event of an FTN failure, the replacement protocol issues a new replacement from the pool of available droplets. This replacement is always placed at the head of the chain. Any new flows can be propagated to this new replacement node via passive replication. Simultaneously, existing flows can be replicated via an active replication protocol between the active tail node and the newly replaced head node. Active and passive replication can occur simultaneously, with passive replication of new entry flows taking precedence over active replication of old entry flows.

[0250] New node placement

[0251] There are three options for placing a new FTN in the event of a failure of another FTN. The replacement node can be located at the head, tail, or somewhere in the middle of the chain. Placing a completely new node at the tail of the chain will incur a performance overhead since all flows will be new to this new tail. Placing the replacement node in the middle, the existing node takes over the role of the tail node, avoiding any immediate performance overhead on known flow entries. Placing the replacement node at the head of the chain achieves all the benefits of the middle placement scenario while also accounting for successive tail node failures. For example, if Figure 4 The following figure shows a three-node chain with FTNs T1, T2, and T3. In this scenario, if FTN-T3 fails, FTN-T2 will soon fail. In the case of a middle replacement, after FTN-T3 fails, the new FTN-T4 is placed in the middle of the chain. Now that FTN-T2 fails, FTN-T4 becomes the new tail. Since FTN-T4 is a brand new FTN, performance overhead is incurred. In the case of a head replacement, the successive failures of FTN-T3 and FTN-T2 cause FTN-T1 to take over as the new tail. Since FTN-T1 has the same flow entry table as FTN-T3 and FTN-T2, there is no performance overhead.

[0252] FTN fault scenarios

[0253] Any single FTN can fail in three locations. Figure 17 The following FTN failure scenarios are shown, including a head node failure 1710, an intermediate node failure 1720, and a tail node failure 1730. As shown, these locations are the head, somewhere in the middle, or the tail. When the tail node 1740 fails, the previous node 1750 becomes the new tail. When an intermediate node 1750 fails, the previous node 1760 and the next consecutive node 1740 are connected. Thus, with the failure of the head node 1760, the new replacement node 1770 becomes the new head. Similarly, in the event of multiple failures in the chain, each node will be handled individually according to the above workflow.

[0254] If an FTN failure occurs during scale-out / scale-out in the successor chain, a replacement FTN is placed at the original head of the chain. The pseudo-head of the chain will always be the topmost node. This is done to enable the passive replication protocol to correctly propagate decisions from the old chain to the new chain. During scale-out / scale-out, placing the replacement node at the topmost level of the chain will result in overwriting old linked list entries due to passive replication. FTN failures in the predecessor chain are handled by the replacement protocol without requiring any action on behalf of the successor chain. Since replication is initiated between the tail of the predecessor chain and the head of the successor chain, any non-tail failures will not affect replication. In the event of a tail failure, replication will proceed between the newly allocated tail of the predecessor chain and the head of the successor chain.

[0255] Active replication to replacement FTN

[0256] With passive replication alone, existing flow entries are never replicated to the replacement node. Any flow entries already known to the tail node are simply bounced back to the FWD node with rewritten external packet information. In the scenario where the replacement node becomes the tail node, all flow entries that may have previously existed in the failed node must be re-determined. With active replication, hop overhead is avoided by proactively updating the replacement node with all unknown flow entries currently residing in the tail node. Active replication never overwrites existing flow entries. This is done to preserve new flow replication that may occur simultaneously with passive replication. Figure 18 The active replication process in the exemplary embodiment is shown. Specifically, Figure 18 An example of active replication that occurs when replacement node FTN-3 1800 is added to chain 0, which includes nodes FTN-1 1810 and FTN-2 1820, is outlined.

[0257] like Figure 18 As shown, the replication process includes the following steps:

[0258] Step 1: FTN-2 1820 performs active replication and sends the contents of its table to FTN-3 1800. Note that the table entry (4, D, Open) 1840 may have been previously propagated by the passive replication protocol. Active replication only writes table entries that are new to FTN-3 1800.

[0259] Step 2: FTN-3 1800 writes all unknown entries into its own table and sends the contents of its table to the next node in the chain (FTN-1 1810).

[0260] Step 3: FTN-1 1810 writes all unknown entries into its own table and sends the contents of its table to the end of the chain (FTN-2 1820).

[0261] Step 4: FTN-2 1820 knows all the entries and the active replication is completed.

[0262] To handle failures during active replication, a node must first check its own health and the receiving node's health before sending any updates. If both nodes are healthy but the update fails, the sending node will retry a set number of times. If the maximum number of retries is reached or any node becomes unhealthy, the replacement protocol will be activated. Consequently, the active replication protocol will abort its current task and restart its process from the current head and tail of the chain.

[0263] Operator Effect

[0264] The functions of several operators have been described above. Figure 19 Summarize what each operator does. Figure 19 The management plane architecture 1900 in an exemplary embodiment is shown, where operators 1910 are organized within a Kubernetes cluster controlled by the Kubernetes control node (Kubectl) 1920. Operators 1910 expose interfaces and customer resource definitions to users and administrators via the Kubernetes API server 1930, which provides an internal interface for operators 1910 to communicate with each other during management plane workflows. Operators 1910 also communicate with D-Nodes 210 via the nomad agent / consul 1940, which includes middleware that provides functionality through plugins, including: a placement operator 1942 that retrieves spec sheets from the extension endpoint operator and places them on D-Nodes 210; health checks 1944; discovery 1946 for dynamically adding and removing D-Nodes 210 from the system; and autoscaling 1948.

[0265] DFT Operator 1950: The DFT Operator 1950 exposes interfaces for managing the lifecycle of DFT objects. These functions include generating a Maglev hash table, creating chain objects with corresponding replicas, and actively repartitioning chains. These functions are then organized into workflows to accomplish tasks such as creating, updating, and deleting DFT objects. The lifecycle of a DFT object is managed by three workflows, each triggered by a state change in the DFT object itself. These workflows are creation, update, and deletion. When a DFT is created, the DFT object generates a Maglev hash table and calls Chain Operator 1955 to create a chain object with multiple replicas. The DFT update workflow is triggered by a scale-out or scale-out event. In these scenarios, the DFT Operator 1950 generates a new Maglev hash table and manages chain expansion and repartitioning. Finally, a DFT delete event triggers a workflow that deletes all chains associated with the given DFT, as well as the DFT object itself.

[0266] Chain Operator 1955: The functionality exposed by Chain Operator 1955 manages the creation of FTN nodes, their placement within the chain, and the active replication protocol. Upon chain creation, Chain Operator 1955 calls the FTN Operator to create multiple FTN replicas and assign them positions within the chain. When the chain's FTNs are ready, Chain Operator 1955 notifies all forwarding device (FWD) operators in the forwarding layer 410 of these changes. Chain updates are triggered by active replication or scale-out / scale-out. In this scenario, Chain Operator 1955 relocates the FTNs and facilitates active replication between FTNs. Finally, upon chain deletion, all child FWDs of the chain are deleted, and all FWDs in the forwarding layer 410 are notified of the deleted chain.

[0267] Together with FTN node behavior, Chain Operator 1955's chain enhancements facilitate repartitioning and rebalancing flow entries across multiple chains without requiring external threads to replicate flow entries. Rebalancing and partitioning at the flow level prevents all flows from noisy tenants from being concentrated in a single chain. Unlike traditional approaches, which typically require active replication of tenant-specific state between nodes, Chain Operator 1955's chain enhancements reduce management overhead and the impact of noisy neighbors on DFT.

[0268] FTN operator 1960: FTN operator 1960 via nomad / consul cluster 227 ( Figure 2) interacts with droplet-d221 to perform actions such as issuing FTN nodes and initializing eBPF maps. The FTN workflow then invokes these actions to complete the process of creating, issuing, and deleting FTN nodes. During FTN creation, the FTN object is assigned the droplet object to be placed, the next node in its chain, and the chain object to which it belongs. It also initializes the eBPF map 340 on its assigned droplet. The FTN object update workflow updates the eBPF map 340 of the FTN object and invokes the middlebox operator 1975 to load any M-XDP programs 320. Finally, the FTN removal workflow unloads / removes all associated M-XDP programs 320 and eBPF maps 340 from the FTN's assigned droplet.

[0269] Extended Endpoint Operator 1965: The Extended Endpoint Operator 1965 exposes interfaces for managing extended endpoint objects. These include functions for placing and configuring middleboxes, placing extended endpoints on FWDs, and scaling extended endpoints up / down on FWDs. Upon extended endpoint creation, the Extended Endpoint Operator 1965 will provision middleboxes to droplets on FWDs and FTNs according to the specified configuration. Upon extended endpoint update, the topology of child middleboxes across droplets will be updated to reflect the new configuration. Finally, upon deletion, all middleboxes specified in the extended endpoint configuration and their associated M-XDP programs 320 will be deleted.

[0270] FWD Operator 1970: The FWD Operator 1970 is responsible for creating and deleting forwarding layer 410 FWDs on droplets. The FWD Operator 1970 performs actions such as issuing forwarding layer 410 FWDs and updating data stores after issuance. The FWD workflow calls functions within the operator class to manage the lifecycle of FWD nodes. When creating a forwarding layer 410 FWD, the FWD object is assigned a droplet and any necessary eBPF maps 340 are initialized on the droplet. The FWD object update workflow can be used to scale the number of forwarding layer 410 FWDs in the cluster or to place middleboxes on their assigned droplets. Finally, the FWD deletion workflow uninstalls / deletes all associated M-XDP programs 320 and eBPF maps 340 from the FWD's assigned droplet.

[0271] Middlebox Operator 1975: The middlebox operator 1975 exposes an interface for interacting with middlebox objects. These functions include placing middlebox objects on droplets and loading / unloading M-XDP programs 320 on droplets. When created, a middlebox object loads its designated M-XDP program 320 onto its designated droplet. Similarly, when deleted, the M-XDP program 320 is unloaded from the middlebox's assigned droplet.

[0272] Droplet Operator 1980: The Droplet Operator 1980 is responsible for managing the lifecycle of a droplet, such as its creation and deletion. The Droplet Operator 1980 is also responsible for exposing an interface for loading any M-XDP program 320 onto it. Upon droplet creation, the droplet object is assigned to the host node and initialized with host-specific information (such as IP address, MAC address, etc.). Upon droplet deletion, the droplet object is simply removed from the data store as a free resource.

[0273] Exemplary Cases

[0274] Now refer to Figures 20 to 23B Describe a typical use case for the above system.

[0275] East / West traffic

[0276] Figure 20 is an exemplary use case showing east / west traffic in a data center, where the destination IP address is for an extension endpoint and out.daddr is one of the FWD IP addresses of the extension endpoint. Figure 20 The following example shows traffic between two VMs 10.0.0.2 (2000) and 10.0.0.3 (2010) in the same virtual network (i.e., sharing the same VNI in the encapsulated packets):

[0277] 1. At 2020, VM 10.0.0.2 2000 sends a packet through its bouncer to VM 10.0.0.3 2010. The bouncer itself is an extended endpoint, so the host selects one of its FWDs (for example, FWD 2030) to send the packet.

[0278] 2. At 2040, the FWD 2030 adds an NSH header 2042 to the message and sends the message to the FTN trailer 2050 of the DFT in the chain corresponding to the flow.

[0279] 3. At 2060, the FTN tail 2050 of the DFT tail calls the bouncer M-XDP 2052 (or the separator 2054 if the two VMs 2000 and 2010 are in separate networks). The bouncer 2052 determines the outer IP address of the encapsulated packet as the endpoint host.

[0280] 4. At 2070 , since the decision of bouncer 2052 and splitter 2054 is static (the same destination IP address will always have the same external IP), there is no need to persist the decision in DFT and FTN tail 2050 responds to FWD 2030 immediately.

[0281] 5. At 2080, FWD 2030 caches the flow entry, strips off the NSH header 2042, rewrites the outer IP header, and sends the packet to host VM 2010 at 10.0.0.3.

[0282] The reverse flow follows the same procedure, with the remaining flow packets being processed directly by FWD 2030 without sending packets to the DFT. In the Mizar cloud computing system, the FWD is actually collocated with the VM host. Using the DFT for management plane scale in this way has an inherent advantage. Because the FWD is aware of the DFT partitions through consistent hashing, introducing a new VM into the system only requires programming a single replica of the FTN, tightening the management plane complexity to O(1). This means that no matter how much data is provided for processing, the association algorithm always completes in constant time.

[0283] North / South traffic

[0284] Figure 21 Figure 2 is an example use case showing north / south forwarding traffic for a virtual machine (VM) 2100, where the destination IP address is for another extended endpoint, in.daddr is the IP address of the extended endpoint, and out.daddr is one of the FWD IP addresses of the extended endpoint. In this example, an external client 72.0.0.1 2102 connects to an extended endpoint 2104 that acts as a load balancer. The public IP address of the extended endpoint 2104 is 52.0.0.2. The extended endpoint 2104 has multiple VMs configured as backends for the extended endpoint 2104. The following message sequence is shown:

[0285] 1. The provider router 2110 sends the client packet to one of the extended endpoint's FWDs 2120. The extended endpoint operator should have configured the router 2110 to balance traffic from the Internet 2112 to the FWDs 2120 via ECMP routing on the router 2110.

[0286] 2. At 2160, since the packet is not encapsulated, FWD 2120 encapsulates the packet using encapsulation module 2122 and sets its vni = 0, indicating an encapsulated substrate packet. FWD 2120 adds an NSH header and passes the packet to ACL M-XDP 2124. If the ACL rule allows the packet, ACL M-XDP 2124 sends the packet to the end of the chain 2130 of the flow.

[0287] 3. At 2165, the P-XDP tail of the tail FTN 2130 calls the dynamic network address translator (DNAT) M-XDP 2132, which determines the extended endpoint vni = 0xff, selects one of its backends as in.daddr = 10.0.0.3, and adds vni and in.daddr to the NSH header.

[0288] 4. At 2170, DNAT M-XDP 2132 then tail calls bouncer M-XDP 2134 through load balancer 2136, which determines that VM 10.0.0.3 (2100) is hosted at out.daddr=host-ip and adds this information to the NSH header.

[0289] 5. 6. 7. At 2175, 2180, and 2185, bouncer M-XDP 2134 then sends the packet to the head 2138 and copy 2140 of the DFT until the packet returns to the tail FTN 2142. The DFT 2150, which includes FTNs 2132-2142, maintains the forward and reverse flow entries.

[0290] 8. 9.T 2190 and 2195, tail FTN 2142 returns the packet to FWD 2120, FWD 2120 strips the NSH header, applies the changes in the flow table entry (modifies vni, out.daddr, in.daddr), and sends the packet to host 2100 at 10.0.0.3.

[0291] Subsequent flow forwarding packets are processed only through the FWD 2120 (ie, two hops).

[0292] Figure 22 Shown Figure 21 Consider the north / south reverse traffic flow in the example of [ 21]. A reverse packet from VM host 2100 enters one of its bouncer FWDs 2120. In this case, no new decision needs to be made because DFT 2150 has kept the reverse flow incoming during processing of the first packet of the flow. Consider the following sequence of packets:

[0293] 1. At 220 , the VM host 2100 returns the VM traffic to the streaming traffic through one of its bouncer FWDs 2120 .

[0294] 2. At 2210, since this is the first time that the FWD 2120 sees this flow (reverse flow), it looks up the flow table entry via the tail FTN 2152 of the corresponding chain.

[0295] 3. At 2220 , the tail FTN 2152 finds the flow entry for the reverse flow, updates the NSH header, and sends a reply message to the FWD 2120 .

[0296] 4. At 2230 , FWD 2120 rewrites the source IP address to 52.0.0.2, decapsulates the packet using decapsulation module 2240 , and sends it to the service provider router 2110 .

[0297] IPv4 to IPv6 Transmission

[0298] Figure 23A and Figure 23B In one example, IPv4-to-IPv6 transmission is illustrated. In this example, client VM 2300 in VPC1 with an IPv4 address of 72.0.0.4 is routed via an extended endpoint to destination VM 2310 in VPC2 with an IPv6 address of 0:0:0:0:0:0:ffff:3400:2 (i.e., 52.0.0.2 in IPv4). Note that client VM 2300 only supports IPv4, while destination VM 2310 only supports IPv6. Consider the following message sequence:

[0299] 1. At 2360, the client VM host 2300 sends a packet with a source address of 72.0.0.4 and a destination address of 52.0.0.2. The client VM host 2300 (private IP address of 172.0.0.8) selects the FWD 2320 of the extended endpoint in Droplet 1 and sends the packet.

[0300] 2. At 2362, FWD 2320 encapsulates the message using encapsulation module 2322 and sets vni = 0. FWD 2320 also adds NSH header 2364 to the message and calls the message tail into ACL M-XDP 2324, where ACL 2324 filters the message according to the configured access rules.

[0301] 3. At 2366, ACL M-XDP 2324 sends the message to the tail FTN node 2330 of the DFT.

[0302] 4. At 2368, FTN node 2330 tail-calls Bouncer M-XDP 2332. Because the destination VM and the client VM are not in the same VPC, Bouncer M-XDP 2332 cannot determine the external IP address of the encapsulated packet from the Bounder routing table (Table 1). Therefore, Bouncer M-XDP 2332 tail-calls Delimiter M-XDP 2334.

[0303]

[0304] 5. At 2370, the separator M-XDP 2334 routing table (Table 2) determines that the vni value is 0x5, which is the vni number of VPC 2. The separator M-XDP 2334 then updates the NSH header with vni = 0x5.

[0305]

[0306] 6. At 2372, Separator M-XDP 2334 tail calls IPTranslator M-XDP 2336. IPTranslator M-XDP 2336 includes the IPv6 mapping (Table 3) and determines the corresponding IPv6 address and port number for 52.0.0.2. IPTranslator M-XDP 2336 uses the same IP address for hosts in the same VPC, but with different port numbers. This approach is useful in IPv6-to-IPv4 translation scenarios (Table 4), as one IPv4 address can be used for many IPv6 addresses, preventing IPv4 exhaustion.

[0307]

[0308]

[0309] 7. At 2374, IPTranslator M-XDP 2336 tail-calls Port Address Translation (PAT) M-XDP 2338. PAT M-XDP 2338 includes the IPv6 port address mapping (Table 5) and determines that the destination VM's in.daddr is 10.0.0.3 and that it is hosted at out.daddr = host IP address. PAT M-XDP 2338 then updates the NSH header with in.daddr = 10.0.0.3 and out.daddr = host IP address. When traffic flows from VPC2 to VPC1 (translating IPv6 to IPv4), the same IPv4 address can be mapped to all VMs in VPC1 via port number, as shown in the IPv4 port address mapping (Table 6).

[0310]

[0311]

[0312] 8. 9. 10. At 2376, 2378, and 2380, PATH M-XDP 2338 sends the message to the head 2340 and copy 2342 of the DFT until the message returns to the tail FTN 2344, as mentioned in the previous example. Both the forward and reverse flow entries are written to the DFT 2350.

[0313] At 2382 and 2384, tail FTN 2344 forwards the packet back to FWD M-XDP 2320. FWD 2320 removes NSH header 2364 and updates the flow entry (vni, out.daddr, in.daddr) in its cache. FWD M-XDP 2320 then sends the packet to VM host 2310 at 10.0.0.3 in VPC2.

[0314] The systems and methods described herein separate connection state tracking from packet forwarding, enabling the forwarding layer to scale independently of the connection state tracking layer. Automatic scaling through chain enhancements eliminates the operational overhead of manually adding new flow tracking nodes. Furthermore, unlike traditional systems that use a three-tier architecture for fixed network functions and flow trackers, this system can be implemented in two layers, minimizing the number of hops required to establish a connection and enabling scaling of its functionality and tracking of different types of state (beyond just flow tracking decisions).

[0315] These systems and methods also provide for collocated middlebox decision making with tail nodes and self-validation capabilities. This approach enables configuration changes to take effect immediately on existing flows without the need for external synchronization mechanisms. These systems and methods also enable modular expansion of the functionality of D-Node 210, which includes several middlebox functions. These modules are distributed and decoupled, and a declarative programming model facilitates interoperability between these distributed and decoupled modules.

[0316] Hardware Implementation

[0317] Figure 24 A general-purpose computer 2400 suitable for implementing one or more embodiments of the methods disclosed herein is shown. The components described above can be implemented on any general-purpose network component, such as computer 2400, that has sufficient processing power, memory resources, and network throughput capabilities to handle the workload placed upon it. Computer 2400 includes a processor 2410 (which may be referred to as a central processor unit (CPU)) that communicates with memory devices including secondary memory 2420, read-only memory (ROM) 2430, and random access memory (RAM) 2440, I / O devices 2450, and network connectivity devices 2460. In an exemplary embodiment, network connectivity devices 2460 also connect processor 2410 to a distributed network service system 2470, as described herein. Processor 2410 can be implemented as one or more CPU chips or as part of one or more ASICs.

[0318] Secondary storage 2420, typically comprised of one or more disk drives and / or tape drives, is used for non-volatile data storage and also serves as an overflow data storage device if RAM 2440 is not large enough to hold all working data. Secondary storage 2420 can be used to store programs that are loaded into RAM 2440 when they are selected for execution. ROM 2430 can be used to store instructions and possibly data read during program execution. ROM 2430 can be a non-volatile memory device that typically has a smaller memory capacity relative to the larger memory capacity of secondary storage 2420. RAM 2440 is used to store volatile data and may be used to store instructions. Access to both ROM 2430 and RAM 2440 is typically faster than access to secondary storage 2420.

[0319] It should be understood that the computer 2400 can execute instructions from a computer-readable non-transitory medium storing computer-readable instructions and one or more processors coupled to the memory, and when executing the computer-readable instructions, the computer 2400 is used to perform the present invention in combination with Figures 1 to 23B Computer-readable non-transitory media include all types of computer-readable media, including magnetic storage media, optical storage media, flash memory media, and solid-state storage media.

[0320] It should also be understood that software comprising one or more computer-executable instructions that facilitate the processing and operations described above in connection with any or all of the steps of the present invention can be installed on and sold with one or more servers or databases. Alternatively, software can be obtained and loaded onto one or more servers or one or more databases in a manner consistent with the present invention, including obtaining the software via physical media or a distribution system, including, for example, from a server owned by the creator of the software or from a server not owned by the creator of the software but used by the creator of the software. For example, the software can be stored on a server for distribution via the Internet.

[0321] In addition, it will be understood by those skilled in the art that the present invention is not limited to the construction and arrangement details of the components set forth in the specification or shown in the drawings in terms of its application. The embodiments herein are applicable to other embodiments and can be practiced or executed in various ways. In addition, it will be understood that the wording and terminology used herein are for descriptive purposes and should not be considered restrictive. The use of "including," "comprising," or "having" and their variations herein is intended to encompass the items listed thereafter and their equivalents as well as other items.

[0322] The components of the illustrative devices, systems, and methods used in accordance with the described embodiments may be implemented at least in part in digital electronic circuitry, analog electronic circuitry, or in computer hardware, firmware, software, or a combination thereof. For example, the components may also be implemented as a computer program product (e.g., a computer program, program code, or computer instructions) tangibly embodied in an information carrier or in a machine-readable storage device for execution by a data processing apparatus (e.g., a programmable processor, a computer, or multiple computers), or for controlling the operation of a data processing apparatus.

[0323] Computer programs can be written in any form of programming language (including compiled or interpreted languages) and can be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. Computer programs can be deployed to be executed in one computer or multiple computers at one site, or distributed at multiple sites and interconnected by a communication network. In addition, the functional programs, codes, and code segments for implementing the systems and methods described herein can be readily understood by programmers in the field to which the present invention belongs to be within the scope of the present invention. The method steps associated with the illustrative embodiments can be performed by one or more programmable processors, thereby executing computer programs, codes, or instructions to perform functions (e.g., operate on input data and generate output). The method steps can also be performed by special-purpose logic circuitry, and the apparatus can be implemented as a special-purpose logic circuitry, such as a field programmable gate array (FPGA) or an ASIC.

[0324] The various illustrative logical blocks, modules, and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed using a general-purpose processor, a digital signal processor (DSP), an ASIC, an FPGA, or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the general-purpose processor may also be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, such as a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in combination with a DSP core, or any other similar configuration.

[0325] For example, processors suitable for executing computer programs include general-purpose and special-purpose microprocessors, as well as any one or more processors of any type of digital computer. Typically, a processor will receive instructions and data from read-only memory or random access memory, or both. The elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer also includes one or more mass storage devices (e.g., magnetic, magneto-optical, or optical disks) for storing data, or is operatively coupled to one or more mass storage devices for storing data to receive data from and / or transfer data to the mass storage devices. Information carriers suitable for embodying computer program instructions and data include various forms of non-volatile memory, including, for example, semiconductor memory devices such as electrically programmable read-only memory or electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory devices, and data storage disks (e.g., magnetic disks, internal hard disks, or removable disks, magneto-optical disks, compact disc ROMs (CD-ROMs), or digital versatile disc ROMs (DVD-ROMs)). The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0326] Those skilled in the art will appreciate that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips referenced in the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or optical particles, or any combination thereof.

[0327] Those skilled in the art will also appreciate that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention. Software modules may reside in random access memory (RAM), flash memory, ROM, EPROM, EEPROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such that the processor can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium may be integral to the processor. In other words, the processor and storage medium may reside in an integrated circuit or implemented as discrete components.

[0328] As used herein, "machine-readable medium" refers to a device capable of temporarily or permanently storing instructions and data, and may include, but is not limited to, random-access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of memory (e.g., EEPROM), and / or any suitable combination thereof. The term "machine-readable medium" should be understood to include a single medium or multiple media (e.g., a centralized or distributed database, or associated cache memory and servers) that can store processor instructions. The term "machine-readable medium" should also be understood to include any medium or combination of multiple media that can store instructions for execution by one or more processors, such that the instructions, when executed by the one or more processors, cause the one or more processors to perform any one or more of the methods described herein. Accordingly, "machine-readable medium" refers to a single storage device or device, as well as a "cloud-based" storage system or storage network comprising multiple storage devices or devices. As used herein, the term "machine-readable medium" does not include signals per se.

[0329] Although several embodiments have been described in detail above, other modifications are possible. For example, the logic flows depicted in the figures do not require the particular order shown or sequential order to achieve the desired results. Additional steps may be provided or deleted from the described flows, and additional components may be added to or deleted from the described systems. Other embodiments are within the scope of the appended claims.

Claims

1. A network service system, characterized in that: include: a connection state tracking layer, comprising first nodes organized into at least one chain, each first node in the chain comprising a modular middlebox, the modular middlebox being linked to at least one other modular middlebox in the chain to implement a network service function for a data packet, the modular middlebox comprising at least one specification table, the at least one specification table determining a sequence of actions for implementing the network service function for the data packet, the connection state tracking layer implementing a distributed flow table (DFT) comprising at least one chain of first nodes; a stateless forwarding layer comprising at least one second node, the at least one second node receiving the data packet, selecting the flow entry from the DFT when a flow entry is available, rewriting a header of the data packet according to the flow entry, and sending the data packet to a first destination based on the flow entry; at least one third node, converting a network service described in a declarative programming language into the at least one specification sheet, and distributing the at least one specification sheet to at least one of the modular middleboxes in the at least one first node chain of the DFT; The modular middlebox includes a plurality of fast data path XDP programs in a local data structure and a shared data structure configuration for processing data packets to implement the network service function.

2. The network service system according to claim 1, wherein: The DFT is implemented in a key-value memory.

3. The network service system according to claim 1, wherein: Each first node in the at least one chain executes a chain replication protocol that replicates state and data packet flow table entries to maintain a consistent flow state across all first nodes in the at least one chain.

4. The network service system according to claim 1, wherein: The at least one third node provides an extended endpoint operator that generates the specification sheet from an extended endpoint configuration written in the declarative programming language. The extended endpoint operator further propagates the specification sheet of the network service function to the modular middleboxes of the at least one first node chain of the DFT to implement the network service function.

5. The network service system according to claim 4, characterized in that: It also includes an extended endpoint, which spans multiple second nodes of the stateless forwarding layer and is automatically expanded according to data message traffic requirements.

6. The network service system according to claim 5, characterized in that: The method further includes a selector configured to map the extended endpoint to at least one second node of the stateless forwarding layer.

7. The network service system according to claim 6, characterized in that: The selector includes at least one of a router, a software switch, an extended Berkeley packet filter (eBPF) program, or a Maglev hash table, and the selector selects the at least one second node of the stateless forwarding layer for the extended endpoint.

8. The network service system according to claim 5, characterized in that: The extended endpoint provides at least one of a load balancer, a network address translator NAT, a deep packet inspector DPI, quality of service QoS, a tunnel, a firewall, a protocol converter IPv6 / IPv4, an access control list ACL, or a combination thereof.

9. The network service system according to claim 1, wherein: The connection state tracking layer and the stateless forwarding layer are configured to scale independently of each other and not affect each other in the event of a failure.

10. The network service system according to claim 1, wherein: The DFT includes a Maglev hash table that maps each first node chain of the DFT to at least one other first node chain of the DFT.

11. The network service system according to claim 1, wherein: Each first node chain of the DFT includes an end node that implements a co-located middlebox function, and the modular middlebox of the end node implements all or part of the message processing function of the network service function described in the at least one specification table.

12. The network service system according to claim 1, wherein: The at least one first node chain of the DFT implements a network service function of the second data message; The at least one second node of the stateless forwarding layer also receives the second data packet, and based on the fact that the second flow table entry of the second data packet cannot be obtained from the DFT, selects the first node chain of the DFT using the Maglev hash algorithm, and sends the second data packet to the tail node of the selected first node chain of the DFT.

13. The network service system according to claim 12, wherein: The at least one second node of the stateless forwarding layer further: receiving a new flow table entry for the second data packet from the selected first node chain of the DFT; Based on the new flow table entry, the second data packet is sent to a second destination.

14. The network service system according to claim 13, wherein: The tail node of the selected first node chain of the DFT retrieves an extended endpoint specification table, executes all modular middleboxes of the selected first node chain of the DFT according to the extended endpoint specification table, copies the extended endpoint specification table within the fast data path (XDP) program of the tail node of the selected first node chain of the DFT, determines a new data flow table entry from processing the second data packet according to the extended endpoint specification table, and provides the determined new data flow table entry to the stateless forwarding layer.

15. The network service system according to claim 1, wherein: Also included is one or more processors that implement a DFT operator that filters and predicts resource usage data collected by the DFT, determines a plurality of chains of XDP programs to be added to or deleted from the DFT, recomputes a Maglev hash table that includes new chain assignments to track predecessor chains of each chain of the XDP program, and propagates the recomputed Maglev hash table to the first node of the DFT.

16. The network service system according to claim 1, wherein: The header of the data packet includes a network service header, the network service header including metadata conveying changes required by the stateless forwarding layer for a DFT operation to be applied to the header of the received packet.

17. A method for implementing a network service, characterized in that: include: Converting a network service described in a declarative programming language into at least one specification sheet; Distributing the at least one specification table to modular middleboxes in nodes of a connection state tracking layer, the connection state tracking layer comprising first nodes organized into at least one chain, the modular middleboxes linked to at least one other modular middleboxes to implement a network service function for data packets, the connection state tracking layer implementing a distributed flow table (DFT) comprising the at least one first node chain, wherein the modular middleboxes comprise a plurality of fast data path (XDP) programs in a local data structure and a shared data structure configuration for processing data packets to implement the network service function; Processing the data message according to the action sequence specified in the at least one specification table to implement the network service function of the data message to generate a flow table entry; The data packet is sent to a destination based on the flow table entry.

18. The method according to claim 17, characterized in that The method further includes receiving the data packet when the flow entry is available, selecting the flow entry from the DFT, and rewriting a header of the data packet according to the flow entry.

19. The method according to claim 17, wherein It also includes, when the flow table entry cannot be obtained from the DFT, selecting a first node chain for processing the data packet from the DFT to determine a new flow table entry, receiving the new flow table entry from the DFT, and sending the data packet to a second destination based on the new flow table entry.

Images