Systems and methods for stateless symmetric forwarding
By using network-level consistent hashing technology to symmetrically route traffic in multi-regional organizations, the asymmetric routing problem caused by multi-boundary routers is solved, and greater scalability and load balancing are achieved, reducing system complexity and storage requirements are reduced. .
Patent Information
- Application Number
- CN202380033762.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-07-19
- Filing Date
- 2023-07-13
- Publication Date
- 2024-11-22
AI Technical Summary
The prior art in multi-regional organizations causes asymmetric routing due to the presence of multiple boundary routers, destroying stateful features such as firewalls and virtual private networks, and requires each boundary router to store the flow state, resulting in complexity and lower flows. Expand.
By routing traffic symmetrically using a network-level consistent hash, avoiding each boundary router storing the flow state, allowing traffic to be routed symmetrically through boundary nodes, dynamically adding or removing boundary routers for load balancing.
It realizes symmetric traffic routing without increasing memory usage, reduces system complexity and storage requirements of boundary routers, and supports larger cluster scaling and better load balancing.
Smart Images

Figure CN119013953A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates generally to communication networks, and more particularly to systems and methods for stateless symmetric forwarding in computer networks. Background Art
[0002] A multi-region fabric (MRF) includes regional networks (or overlays) interconnected by software-defined wide area network (SD-WAN) routers called border routers. The border routers provide inter-regional connectivity by interconnecting the regional networks or overlays to a common core network. Typically, there are multiple border routers hosted as a cluster in a regional point of presence (POP) or colo site, and the multiple border routers operate in a fully active mode. Because there are multiple border routers in each cluster, packets can be routed asymmetrically through different border routers in the cluster. BRIEF DESCRIPTION OF THE DRAWINGS
[0003] For a more complete understanding of the present disclosure and other features and advantages thereof, reference is now made to the following description taken in conjunction with the accompanying drawings, in which:
[0004] Figure 1 An example computer network providing stateless symmetric forwarding of packets through a cluster including a plurality of border routers is shown;
[0005] Figure 2 A cluster according to one embodiment of the present disclosure is shown in more detail;
[0006] Figure 3 An example computer network providing stateless symmetric forwarding of packets through a cluster connected to two different types of computer networks according to some embodiments of the present disclosure is shown;
[0007] Figure 4 An example method for stateless symmetric forwarding of packets through a cluster is shown according to some embodiments of the present disclosure; and
[0008] Figure 5 An example method for stateless symmetric forwarding of packets through a cluster connected to two different types of computer networks is shown in accordance with some embodiments of the present disclosure. DETAILED DESCRIPTION
[0009] Overview
[0010] According to one embodiment of the present disclosure, a system provides stateless symmetric forwarding of packets in a computer network. The system includes a memory and a processor. The system determines the cluster state of multiple border routers in a cluster. Then, the system transmits the cluster state to at least one branch node in the computer network. The system generates a network-level consistent hash based on the cluster state, and uses the network-level consistent hash to route a first packet through a first border router among multiple border routers in the cluster. After the first packet is sent through the first border router, the system uses the network-level consistent hash to route a second packet through a first border router among multiple border routers in the cluster.
[0011] According to one embodiment of the present disclosure, a method provides stateless symmetric forwarding of packets by interconnecting a cluster of two different types of computer networks. The method includes: determining the cluster state of multiple border routers in the cluster, and transmitting the cluster state to at least one branch node. The method includes: generating a network-level consistent hash based on the cluster state, and routing a first packet from a first computer network through a first border router among multiple border routers in the cluster using the network-level consistent hash. After the first packet is sent to the second computer network through the first border router, a second packet received by the second border router is transferred from the second computer network by routing the second packet from the second border router in the cluster to the first border router in the cluster using the network-level consistent hash.
[0012] Certain embodiments of the present disclosure may provide one or more technical advantages. A technical advantage of one embodiment may allow a border router to reduce memory usage by not requiring each border router in a cluster to store the flow state of each data flow through the cluster in order to symmetrically route each flow. In contrast, the present disclosure uses less memory by routing each flow according to network-level consistent hashing. Because certain embodiments of the present disclosure do not require the storage of each flow state in each border router, these embodiments are less complex, easier to use, easier to debug, and will have optimal memory usage. Certain embodiments may allow high flow expansion through border routers because the memory usage of border routers is reduced. Some embodiments may allow dynamic addition and deletion of border routers in a cluster, which makes these embodiments easier to operate than traditional cluster designs. Another technical advantage of certain embodiments of the present disclosure is that the use of network-level consistent hashing can achieve better load balancing when more border routers are dynamically added or deleted from a cluster. One or more other technical advantages will be apparent to those skilled in the art from the drawings, descriptions, and claims included herein.
[0013] Example Embodiments
[0014] A multi-region fabric (MRF) includes regional networks (or overlays) interconnected by software-defined wide area network (SD-WAN) routers called border routers. Border routers provide inter-regional connectivity by interconnecting regional networks or overlays to a common core network. Typically, there are multiple border routers hosted as a cluster in a regional point of presence (POP) or colo site, and the multiple border routers operate in a fully active manner. Because there are multiple border routers in each cluster and there are multiple paths end-to-end through different border routers, packets can be routed asymmetrically through different border routers in the cluster. Such asymmetrically routed traffic flows may disrupt stateful features, such as firewalls, virtual private networks (VPNs), or deep packet inspection (DPI). These features require seeing both directions of the flow when implemented, for example, from the client to the server and vice versa. Typically, the asymmetric routing traffic forwarding problem of stateful applications is solved by storing flow state synchronization that provides flow-owner border router information so that non-owner border routers can transfer asymmetrically routed traffic to the correct owner border router. However, this solution is very complex and requires additional memory to store flow states in non-owner border routers, which in turn results in low flow scaling, also for clusters of multiple nodes.
[0015] The present disclosure describes embodiments that provide solutions to the above-described problems by using network-level consistent hashing to symmetrically route traffic through border routers in a cluster of multiple border routers. The present disclosure solves many of the problems of relying on stateful symmetric routing to solve asymmetric routing problems. The present disclosure provides systems and methods that allow traffic flows to be symmetrically routed through border nodes without requiring the state of the flows to be stored in each border router in the cluster. Because the present disclosure is stateless, symmetric traffic flows through border routers in a cluster can be achieved with little or no impact on memory. Therefore, the present disclosure allows for greater expansion of clusters due to less complexity and less memory usage.
[0016] Figure 11 is an illustration of an example computer network 100 that provides stateless symmetric forwarding of packets through a cluster containing multiple border routers. In the illustrated embodiment, the network 100 includes two regional networks and a core network labeled "OMP Core". Each regional network is interconnected to the core network via two clusters 110. Each cluster 110 includes multiple border routers 120. The SD-WAN overlay interconnects multiple edge routers 130 with each cluster 110. In the illustrated embodiment, the branch node 180 is connected to one or more edge routers 130. According to the illustrated embodiment, the cluster state 140 of each cluster 110 is transmitted to the one or more edge routers 130 and the one or more clusters 110 via one or more controllers 150. In the illustrated embodiment, the initial flow 160 of traffic is routed from the first branch node 180a through the first edge router 130a. Then, the initial flow 160 of traffic is routed from the first edge router 130a through the first border router 120a, then through the second border router 120c, and then through the second edge router 130d to the second branch node 180b. Upon receiving the initial flow 160 of traffic, the second branch node 180b routes the return flow 170 from the second branch node 180b through the second edge router 130d. The return flow 170 of traffic is then routed from the second edge router 130d through the second border router 120c, then through the first border router 120a, and then through the first edge router 130a to the first branch node 180a.
[0017] The border router 120 represents any suitable network device that facilitates communication between endpoints in the first network and endpoints in the second network. The border router may include a border router and an edge router. The border router may be interconnected to endpoints in each network through the public Internet, a multi-protocol label switching (MPLS) virtual private network (VPN), a software defined wide area network (SD-WAN), etc. In some embodiments, the border router represents any suitable network device that facilitates communication between endpoints in the first area of the network and the second area of the network. In some embodiments, the border router represents any suitable network device that facilitates communication between endpoints in the first network and the core network. In these embodiments, the core network may include an overlay management protocol (OMP) core network.
[0018] Cluster 110 represents any grouping of two or more border routers 120. Cluster 110 may include colo sites, regional point of presence (POP) sites, data centers, Internet service provider (ISP) POP sites, Internet exchange points, etc. In some embodiments, cluster 110 is operated by an ISP. In other embodiments, cluster 110 is operated by a single organization within a single network. In some embodiments, cluster 110 is located between two different networks. In other embodiments, cluster 110 is located between different areas of the same network. In some embodiments, cluster 110 includes multiple border routers 120 in one device (e.g., a server). In other embodiments, cluster 110 includes multiple border routers 120 located in separate devices but in the same location.
[0019] The edge router 130 represents any suitable network device that facilitates communication between one or more branch nodes 180 and one or more endpoints in a computer network. The edge router 130 may include an edge router, an access router, a branch router, etc. The edge router 130 may be interconnected to endpoints in a network through the public Internet, an MPLS VPN, an SD-WAN, an SD-WAN overlay, a WAN, a LAN, etc. The edge router 130 may communicate with the branch node 180 through a wired or wireless connection. The edge router 130 may communicate via the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), or any other suitable network protocol.
[0020] The branch node 180 represents any suitable device capable of connecting to the network 100. The branch node 180 may include other routers, servers, end-user devices (e.g., laptops, phones, tablet computers), and any other suitable device capable of connecting to the network. The branch node 180 may include one or more network interfaces, at least one processor, and memory interconnected by a system bus and a power supply. In some embodiments, the branch node 180 represents a device capable of running an application or application group on the computer network 100, and includes a general-purpose computing device (e.g., a server, a workstation, a desktop computer, etc.), a mobile computing device (e.g., a laptop computer, a tablet computer, a mobile phone, etc.), a wearable device (e.g., a watch, glasses or other head-mounted display (HMD), a headset device, etc.), etc. Branch nodes 180 may also include Internet of Things (IoT) devices or devices, such as agricultural devices (e.g., livestock tracking and management systems, watering equipment, unmanned aerial vehicles (UAVs), etc.); connected cars and other vehicles; smart home sensors and devices (e.g., alarm systems, security cameras, lighting, appliances, media players, heating, ventilation and air conditioning (HVAC) devices, utility meters, windows, automatic doors, doorbells, locks, etc.); office devices (e.g., desk phones, copiers, fax machines, etc.); medical devices (e.g., pacemakers, biosensors, medical devices, etc.); industrial devices (e.g., robots, factory machinery, construction devices, industrial sensors, etc.); retail devices (e.g., vending machines, point of sale (POS) devices, radio frequency identification (RFID) tags, etc.); smart city devices (e.g., street lights, parking meters, waste management sensors, etc.); transportation and logistics devices (e.g., turnstiles, rental car trackers, navigation equipment, inventory monitors, etc.), etc.
[0021] Cluster state 140 represents any suitable data type that indicates information about cluster 110. Cluster state 140 may include an integer value, a string, or any other data type that can convey information about cluster 110. Figure 1In the two-node fully active cluster shown, cluster state 140 may include data such as "BR11 is active" and "BR12 is active", where "BR11" represents one of the border routers 120 in cluster 110 and "BR12" represents another border router 120 in cluster 110. In some embodiments, cluster state 140 may include data or information indicating that the cluster state has changed. The cluster state may change when a border router 120 in cluster 110 is added or deleted from cluster 110. For example, if a border router 120 is removed from cluster 110, cluster state 140 may include data such as "BR12 is removed from cluster." In some embodiments, cluster state 140 may include data or information indicating only the number of active border routers 120 in cluster 110. For example, cluster state 140 may include data such as "2".
[0022] Controller 150 represents any suitable network device that facilitates establishing and maintaining a network topology and making decisions about where traffic flows in network 100. Controller 150 may include one or more physical or virtual network controller devices. Controller 150 may include physical or virtual network controller devices such as SD-WAN vSmart controller and the like. In some embodiments, the controller 150 can establish a secure connection to each device in the network 100 and distribute routing and policy information via a control plane protocol. The control plane protocol may include Overlay Management Protocol (OMP), Open Shortest Path First (OSPF), Intermediate System to Intermediate System (IS-IS), Border Gateway Protocol, Protocol Independent Multicast (PIM), Internet Group Management Protocol (IGMP), Internet Control Message Protocol (ICMP), Address Resolution Protocol (ARP), Bidirectional Forwarding Detection (BFD), Link Aggregation Control Protocol (LACP), etc. In some embodiments, the controller 150 is capable of transmitting the cluster state 140 of the cluster 110 in the network 100 to any device in the network 100. In some embodiments, the controller 150 transmits the cluster state 140 only to the edge router 130 and the border router 120 in the cluster 110. In some embodiments, the controller 150 can operate as a route reflector. The controller 150 can also orchestrate secure connectivity between two or more devices in the network. For example, in some embodiments, the controller 150 can distribute key information between (one or more) network devices. This allows the network to support secure network protocols or applications (eg, Internet Security Protocol (IPSec), Transport Layer Security Protocol (TLS), Secure Shell Protocol (SSH), etc.) without Internet Key Exchange (IKE) and achieves scalability of the network.
[0023] The initial flow 160 represents any suitable flow of data packets, traffic, or messages that can be exchanged between devices of the computer network 100 from the first branch node 180a to the second branch node 180b. The initial flow 160 can be exchanged on a link using a predefined network communication protocol (e.g., Transmission Control Protocol / Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Asynchronous Transfer Mode (ATM) protocol, Frame Relay protocol, or any other suitable protocol). In the illustrated embodiment, the initial flow 160 is exchanged on a link of devices in the network 100 using the TCP protocol that performs the first step of the TCP three-way handshake. In particular, in the illustrated embodiment, the initial flow 160 uses the TCP syn message type, which is used to initiate and establish a connection between the first branch node 180 and the second branch node 180.
[0024] The return flow 170 represents any suitable flow of data packets, traffic, or messages that can be exchanged between devices of the computer network 100 from the second branch node 180b to the first branch node 180a in response to receiving the initial flow 160. The return flow 170 can be exchanged on the link using a predefined network communication protocol (e.g., TCP / IP, UDP, ATM protocol, frame relay protocol, or any other suitable protocol). In the illustrated embodiment, the return flow 170 is exchanged on the link of devices in the network 100 using the TCP protocol that performs the second step of the TCP three-way handshake. In the illustrated embodiment, the return flow 170 uses the TCP syn-ack message type, which is used to initiate and establish a connection between the second branch node 180b and the first branch node 180a.
[0025] In an exemplary embodiment of operation, controller 150a determines cluster state 140a of border routers 120a and 120b in cluster 110a. For example, in the illustrated embodiment, cluster state 140a in cluster 110a is two active border routers 120a and 120b. Controller 150a transmits cluster state 140a to edge routers 130a to 130c. Similarly, controller 150b also transmits cluster state 140a to border routers 120c and 120d in cluster 110b. Controller 150c determines cluster state 140b of border routers 120c and 120d in cluster 110b. Controller 150c transmits cluster state 140b to edge routers 130d to 130f. Similarly, controller 150b also transmits cluster state 140b to border routers 120a and 120b. Each edge router 130a to 130f and border router 120a to 120d then generates a consistent hash table based on the cluster state 140 transmitted to them by the controller 150a to 150c. The process of generating a consistent hash table based on the cluster state 140 is discussed in more detail below. According to some embodiments, when an incoming packet is taken as input, each consistent hash table routes the packet to a specific owner border router 120 in the cluster 110 based on consistent hashing.
[0026] In an exemplary embodiment of operation, initial flow 160 begins when edge router 130a receives a packet from branch node 180a. Edge router 130a uses the five-tuple of the initial flow as a key to perform a consistent hash lookup based on a consistent hash table generated by edge router 130a. Edge router 130a uses the result of the consistent hash lookup to force the routing of initial flow 160 to border router 120a of cluster 110a. Similarly, border router 120a uses the five-tuple of the initial flow as a key to perform a consistent hash lookup based on a consistent hash table generated by border router 120a. Border router 120a uses the result of the consistent hash lookup to force the routing of initial flow 160 to border router 120c of cluster 110b. Border router 120c uses the five-tuple of the initial flow as a key to perform a consistent hash lookup based on a consistent hash table generated by border router 120c. A five-tuple refers to a set of five different values comprising a TCP / IP connection, including a source IP address, a source port number, a destination IP address, a destination port number, and a protocol in use. The border router 120c uses the result of a consistent hash lookup to force the routing of the initial flow 160 to the edge router 130d. The edge router 130d can then route the initial flow 160 to a destination device, which will be a branch node 180b in the illustrated embodiment. In some embodiments, the five-tuple of the initial flow 160 is based on one of the packets of the initial flow 160.
[0027] Still according to an exemplary embodiment of operation, when edge router 130d receives a TCP syn-ack response message from branch node 180b, return flow 170 begins. Edge router 130d uses the five-tuple of the return flow as a key to perform a consistent hash lookup according to the consistent hash table generated by edge router 130d. Edge router 130d uses the result of the consistent hash lookup to force the routing of return flow 170 to the border router 120c of cluster 110b. Using a consistent hash lookup will cause the return flow to be symmetrically routed through the same border router 120c used by the initial flow 160. Similarly, border router 120c uses the five-tuple of the return flow as a key to perform a consistent hash lookup according to the consistent hash table generated by border router 120c. Border router 120c uses the result of the consistent hash lookup to force the routing of return flow 170 to the border router 120a of cluster 110a. The border router 120a uses the five-tuple of the return flow as a key to perform a consistent hash lookup according to the consistent hash table generated by the border router 120a. The border router 120a uses the result of the consistent hash lookup to force the routing of the return flow 170 to the edge router 130a. The edge router 130a then routes the return flow 170 to the destination device, which will be the branch node 180a in the illustrated embodiment.
[0028] The consistent hashing technique described in the exemplary embodiment is such a hashing technique that allows an average of only n / m keys to be remapped when the hash table size is resized (e.g., when a border router 120 is added to a cluster 110), where n is the number of keys and m is the number of border routers 120 in the cluster 110. Traditionally, a change in the size of the hash table will cause many keys to be remapped because the hash function is a modular operation. The consistent hashing method solves this problem because in the consistent hashing algorithm, when a border router 120 in a cluster 110 is removed, the vast majority of keys maintain their previous border router 120 assignments, and the addition of the nth border router causes 1 / n portions of the keys to be mapped to border routers 120 that are different from the previous mappings. In some embodiments, let hashKey(x) and hashBR(x) be hash functions for the keys of the initial flow 160 or the return flow 170 and the border router 120. The hash function can be any suitable hash algorithm that can provide a hash value. The hash function can include message digest 5 (MD5), secure hash algorithm (SHA) 1 and 2, etc. A binary search tree can be used to maintain the ID of the border router 120 in the cluster or hash (where ID is equal to hashBR(x)%360, and x=border router 120 identifier, such as its IP address), and find a successor or minimum value in the binary search tree. In some embodiments, when a new key x of the initial flow 160 or the return flow 170 needs to be hashed according to the consistent hash table, let B be equal to hashKey(x)%360. In order to assign the key x to a specific border router, find the successor B in the binary search tree of the ID of the border router 120. If B is greater than the border router ID in the binary search tree, the key is assigned to the border router with the smallest ID value in the binary search tree. In some embodiments, in order to insert an additional border router Y into the cluster 110, let P be equal to hashBR(Y)%360. Then, the flow whose key hash is less than P is moved from the border router 120 whose ID is the successor of P. If P is the largest ID, flows whose hash of their keys is less than P are moved from the border router with the smallest ID to P. In some embodiments, when a border router is to be removed from cluster 110, consistent hashing will look for a successor of P in a binary search tree and move flows from the successor. If P has no successor in the binary search tree, the flow is moved to the smallest border router represented in the binary search tree. This consistent hashing process serves as an example of how a consistent hashing function will work according to some embodiments of the present disclosure. Consistent hashing according to the present disclosure can be any suitable consistent hashing that can be used as a distributed hashing scheme that operates independently of the number of border routers in the cluster by assigning positions to border routers on an abstract circle or hash ring.While consistent hashing is a preferred embodiment of the present disclosure, some embodiments encompass using any variation of hashing, including more traditional hashing techniques.
[0029] Without departing from the scope of the present disclosure, modifications, additions, or omissions may be made to the network 100. For example, the key for the consistent hash table lookup need not be a five-tuple of the initial flow and the return flow. Instead, the key may be any suitable data type or tuple that can identify a specific traffic pattern of a packet. For example, the key representing the initial flow 160 and the return flow 170 may be a two-tuple, wherein the two-tuple includes a destination media access control (MAC) address and a source MAC address. Alternatively, the key representing the initial flow 160 and the return flow 170 may be a four-tuple including a source IP address, a destination IP address, a virtual LAN (VLAN) tag 1, and a VLAN tag 2. Another modification that may be made to the network 100 is that the controllers 150a to 150c may be combined into a single device. In some embodiments, the network 100 may not include the controller 150, and the border router 120, the cluster 110, the edge router 130, or any suitable device in the network 100 is capable of transmitting the cluster state 140 to the edge router 130 and the border router 120 in the network 100. As another example, the network 100 may include any number of edge routers 130, border routers 120, and branch nodes 180. Another example is that the consistent hash table need not be generated by the edge router 130 or the border router 120; instead, the consistent hash table may be generated by the controller 150, and the controller 150 may transmit the consistent hash table to the edge router 130 or the border router 120. In some embodiments, the edge router 130 and the border router 120 may not receive the consistent hash table and receive routing instructions from the controller 150 because the controller 150 has already generated the consistent hash table and sends the routing instructions based on using the initial flow 160 or the return flow 170 as a key of the consistent hash table.
[0030] Figure 2 A cluster according to one embodiment of the present disclosure is shown in more detail. In the illustrated embodiment, the computer network 200 includes a cluster 210 that interconnects an SD-WAN overlay network 260 and a data center organization network 270. The cluster 210 includes a plurality of edge routers 220. Each edge router 220 in the cluster 210 is communicatively coupled to each other via one or more tunnels 250. The computer network 200 also includes a controller 240. The cluster 210 can transmit a cluster state 230 to the controller 240.
[0031] The border router 220 represents any suitable network device that facilitates communication between endpoints in the first network and endpoints in the second network. The border router 220 may include a border router and an edge router. The border router may be interconnected to endpoints in each network through the public Internet, MPLS VPN, SD-WAN, etc. In some embodiments, the border router represents any suitable network device that facilitates communication between endpoints in the first area of the network and the second area of the network. In some embodiments, the border router represents any suitable network device that facilitates communication between endpoints in the first network and the core network. In these embodiments, the core network may include an OMP core network such as an SD-WAN overlay.
[0032] Cluster 210 represents any grouping of two or more border routers 220. Cluster 210 may include colo sites, regional POP sites, data centers, ISP POP sites, Internet exchange points, etc. In some embodiments, cluster 210 is operated by an ISP. In other embodiments, cluster 210 is operated by a single organization within a single network. In some embodiments, cluster 210 is located between two different networks. In other embodiments, cluster 210 is located between different areas of the same network. In some embodiments, cluster 210 includes multiple border routers 220 in one device (e.g., a server). In other embodiments, cluster 210 includes multiple border routers 220 located in separate devices but in the same location.
[0033] Cluster state 230 represents any suitable data type that indicates information about cluster 210. Cluster state 230 may include an integer value, a string, or any other data type that can convey information about cluster 210. Figure 2 In the illustrated four-node fully active cluster, the cluster state 230 may include data such as "CE1 is active," "CE2 is active," "CE3 is active," and "CE4 is active," where "CE1," "CE2," "CE3," and "CE4" represent each border router 220 in the cluster 210. In some embodiments, the cluster state 230 may include data or information indicating that the cluster state has changed. The cluster state 230 may change when a border router 220 in the cluster 210 is added or deleted from the cluster. For example, if a border router 220 is removed from the cluster 210, the cluster state 230 may include data such as "CE4 is removed from the cluster." In some embodiments, the cluster state 230 may include data or information indicating only the number of active border routers 220 in the cluster 210. For example, the cluster state 230 may include data such as "4."
[0034] Controller 240 represents any suitable network device that facilitates establishing and maintaining a network topology and making decisions about where traffic flows in network 200. Controller 240 may include one or more physical or virtual network controller devices. Controller 240 may include physical or virtual network controller devices such as SD-WAN vSmart controller and the like. In some embodiments, the controller 240 can establish a secure connection to each device in the network 200 and distribute routing and policy information via a control plane protocol. The control plane protocol may include OMP, OSPF, IS-IS, Border Gateway Protocol, PIM, IGMP, ICMP, ARP, BFD, LACP, etc. In some embodiments, the controller 240 is capable of transmitting the cluster state 230 of the cluster 210 in the network 200 to any device in the network 200. In some embodiments, the controller 240 transmits the cluster state 230 to the border router 220 in the cluster 210. In some embodiments, the controller 240 can operate as a route reflector. The controller 240 can also orchestrate the secure connectivity between two or more devices in the network. For example, in some embodiments, the controller 240 can distribute key information between (one or more) network devices. This allows the network 200 to support secure network protocols or applications (e.g., IPSec, TLS, SSH, etc.) without IKE and to achieve network scalability.
[0035] Tunnel 250 represents any suitable point-to-point tunnel that communicatively couples each border router 220 in cluster 210 to each other. Tunnel 250 may include a Generic Routing Encapsulation (GRE) tunnel, a fully meshed GRE tunnel, an IP tunnel, a UDP tunnel, an IPSec tunnel, a Layer 2 Tunneling Protocol (L2TP), a Point-to-Point Tunneling Protocol, a Layer 2 Forwarding (L2F), etc. Tunnel 250 may be formed over a direct communication link to each border router 220 or may be formed over a communication link spanning different networks.
[0036] The SD-WAN overlay network 260 represents any suitable software-defined network topology capable of interconnecting the cluster 210. The SD-WAN overlay network 260 (sometimes also referred to as a network fabric or fabric) is a virtual network topology built on top of an underlying physical network (i.e., an underlying network). The SD-WAN overlay network 260 can create a logical topology for virtually connecting devices built on top of any physical network topology. The SD-WAN overlay network 260 may include GRE or multipoint GRE (mGRE), MPLSVPN or VPLS, IPsec or Dynamic Multipoint Virtual Private Network (DMVPN), CAPWAP (Control and Provisioning of Wireless Access Points), VXLAN, Location / Identifier Separation Protocol (LISP), Overlay Transport Virtualization (OTV) and Application Centric Infrastructure (Cisco ACI TM )wait.
[0037] The data center organization network 270 represents any suitable network topology for implementing a data center and can be interconnected with the cluster 210. The data center organization network 270 may include a software defined networking (SDN) data center organization, a traditional local area network, a spine and leaf architecture, etc. For example, the data center organization network 270 may be Application Centric Infrastructure (Cisco ACI TM ). In some embodiments, the data center organization network 270 may include an SD-WAN overlay network. In some embodiments, the data center organization network 270 is part of the SD-WAN overlay network 260.
[0038] In an exemplary embodiment of operation, a user configures a cluster 210 and border routers 220 within the cluster 210. In some embodiments, the border routers 220 in the cluster 210 can automatically learn peer border routers 220 via the controller 240 using OMP instead of static configuration. The cluster 210 can then establish a point-to-point tunnel 250 to each peer border router 220 that is dynamically learned or statically configured. The cluster 210 can also run a BFD protocol to detect the liveliness of each peer border router 220 and exchange a summary of each border router 220 in the cluster 210. In some embodiments, the summary is referred to as a "cluster node list". Once the BFD algorithm is run and the cluster 210 and each border router 220 have determined that there is a stable state for the border routers 220 in the cluster 210, the cluster 210 will be in a stable state. The cluster 210 can then create a cluster state 230 that will include a list of the border routers 220 of the cluster 210. The cluster 210 may then transmit the cluster state 230 to the controller 240 via the OMP, which in turn reflects the cluster state 230 to the edge routers and branch sites in the SD-WAN overlay network 260. In some embodiments where the cluster 210 is interconnected to the MRF, the cluster 210 announces the cluster state 230 to the access and core regions. In some embodiments, new address families will be defined in the OMP to announce the cluster state 230. When the cluster 210 changes dynamically (e.g., a border router 220 is out of service or a new border router 220 is added to the cluster 210), BFD will quickly detect the change, determine that a change did in fact occur, and will update the cluster state 230 and transmit it to the controller 240 for transmission to the branch sites. In some embodiments, the BFD algorithm used to detect the cluster state 230 of the cluster 210 is configured to be at least 2 to 3 times smaller than the typical update period of the SD-WAN overlay network 260. One advantage of this is that changes in the cluster state 230 of the cluster 210 are detected more quickly and announced to the remote sites before the routing converges. Another advantage in some embodiments is that there will be no black hole traffic or race conditions that would cause a border router 220 to fail or be added to the cluster 210 because the branch sites will see the cluster state 230 before routing converges.
[0039] Without departing from the scope of the present disclosure, modifications, additions, or omissions may be made to the network 200. For example, any number of border routers 220 may be included in the cluster 210. As another example, the cluster 210 may be interconnected to a non-SDN network topology such as a WAN, a LAN, the public Internet, MPLS, etc. Another example is that the data center structure network 270 may include a server. Another example is that the cluster 210 may transmit the cluster state 230 directly to the branch node without passing through the controller 240. Another example is that the controller 240 may announce the cluster state 230 to the data center organization network 270. Another modification may be that the cluster 210 may not use BFD to detect the cluster state 230 of the cluster 210, and may use any suitable networking protocol that can detect failures, removals, or additions in the cluster 210.
[0040] Figure 3 An example computer network is shown that provides stateless symmetric forwarding of packets through a cluster connected to two different types of computer networks according to some embodiments of the present disclosure. In the illustrated embodiment, the network 300 includes two different networks: an SD-WAN overlay network 390 and a data center organization 395. Each network is interconnected by a cluster 310. Each cluster 310 includes multiple border routers 320. The SD-WAN overlay interconnects the edge routers 330 with the clusters 310. In the illustrated embodiment, the branch node 380 is connected to one edge router 330. According to the illustrated embodiment, the cluster state 340 of the cluster 310 is transmitted to the edge node 330 via one or more controllers 350. In the illustrated embodiment, the initial flow 360 of traffic is routed from the first branch node 380 through the edge router 330. Then, the initial flow 360 of traffic is routed from the edge router 330 through the first border router 320b and then through the data center organization 395 to the server 385. Upon receiving the initial flow 360 of traffic, the server 385 routes the return flow 370 from the server 385 through the data center organization 395. Then, the return flow 370 of the traffic is routed from the data center structure 395 to the second border router 320c. Upon receiving the return flow 370, the second border router 320c transfers the return flow 370 to the first border router 320b according to the consistent hash table lookup. Then, the return flow 370 continues to pass through the first border router 320b, and then reaches the branch node 380 through the edge router 330.
[0041] The border router 320 represents any suitable network device that facilitates communication between endpoints in the first network and the endpoints in the second network. The border router 320 may include a border router and an edge router. The border router 320 may be interconnected to the endpoints in each network through the public Internet, MPLS VPN, SD-WAN, etc. In some embodiments, the border router 320 represents any suitable network device that facilitates communication between endpoints in the first area of the network and the second area of the network. In some embodiments, the border router 320 represents any suitable network device that facilitates communication between endpoints in the first network and the core network. In these embodiments, the core network may include an OMP core network.
[0042] Cluster 310 represents any grouping of two or more border routers 320. Cluster 310 may include colo sites, regional POP sites, data centers, ISP POP sites, Internet exchange points, etc. In some embodiments, cluster 310 is operated by an ISP. In other embodiments, cluster 310 is operated by a single organization within a single network. In some embodiments, cluster 310 is located between two different networks. In some embodiments, cluster 310 includes multiple border routers 320 in one device (e.g., a server). In other embodiments, cluster 310 includes multiple border routers 320 located in separate devices but in the same location.
[0043] The edge router 330 represents any suitable network device that facilitates communication between one or more branch nodes 380 and one or more endpoints in a computer network. The edge router 330 may include an edge router, an access router, a branch router, etc. The edge router 330 may be interconnected to endpoints in a network through the public Internet, an MPLS VPN, an SD-WAN, an SD-WAN overlay, a WAN, a LAN, etc. The edge router 330 may communicate with the branch node 380 through a wired or wireless connection. The edge router 330 may communicate via TCP, UDP, or any other suitable networking protocol.
[0044] Branch node 380 represents any suitable device capable of connecting to network 300. Branch node 380 may include other routers, servers, end-user devices (e.g., laptop computers, phones, tablet computers), and any other suitable devices capable of connecting to a network. Branch node 380 may include one or more network interfaces, at least one processor, and memory interconnected by a system bus and a power supply. In some embodiments, branch node 380 represents a device capable of running an application or application group on computer network 300. Branch node 380 may include general computing devices (e.g., servers, workstations, desktop computers, etc.), mobile computing devices (e.g., laptop computers, tablet computers, mobile phones, etc.), wearable devices (e.g., watches, glasses or other HMDs, headset devices, etc.), etc. Branch nodes 380 may also include IoT devices or appliances, such as agricultural devices (e.g., livestock tracking and management systems, watering equipment, UAVs, etc.); connected cars and other vehicles; smart home sensors and devices (e.g., alarm systems, security cameras, lighting, appliances, media players, HVAC devices, utility meters, windows, automatic doors, doorbells, locks, etc.); office devices (e.g., desk phones, copiers, fax machines, etc.); medical devices (e.g., pacemakers, biosensors, medical devices, etc.); industrial devices (e.g., robots, factory machinery, construction devices, industrial sensors, etc.); retail devices (e.g., vending machines, POS devices, RFID tags, etc.); smart city devices (e.g., street lights, parking meters, waste management sensors, etc.); transportation and logistics devices (e.g., turnstiles, rental car trackers, navigation devices, inventory monitors, etc.), etc.
[0045] Server 385 represents any suitable device capable of providing and / or receiving content using any network interconnection protocol to any number of devices on computer network 300. Server 385 may include web servers, database servers, email servers, web proxy servers, domain name system (DNS) servers, file transfer protocol (FTP) servers, file servers, virtual servers, application servers, and dynamic host configuration protocol (DHCP) servers. In some embodiments, server 385 may represent cloud-based resources of network 300. Server 385 may include one or more network interfaces, at least one processor, and memory interconnected by a system bus and power supply. In various embodiments, server 385 may include any number of appropriate servers or other cloud-based resources.
[0046] Cluster state 340 represents any suitable data type that indicates information about cluster 310. Cluster state 340 may include an integer value, a string, or any other data type that can convey information about cluster 310. Figure 3In the three-node fully active cluster shown, cluster state 340 may include data such as "CE1 is active," "CE2 is active," and "CE3 is active," where "CE1," "CE2," and "CE3" represent each border router 320 in cluster 310. In some embodiments, cluster state 340 may include data or information indicating that cluster state 340 has changed. The cluster state may change when a border router 320 in cluster 310 is added or deleted from the cluster. For example, if a border router 320 is removed from cluster 310, cluster state 340 may include data such as "CE1 is removed from cluster." In some embodiments, cluster state 340 may include data or information indicating only the number of active border routers 320 in cluster 310. For example, cluster state 340 may include data such as "3."
[0047] Controller 350 represents any suitable network device that facilitates establishing and maintaining a network topology and making decisions about where traffic flows in network 300. Controller 350 may include one or more physical or virtual network controller devices. Controller 350 may include physical or virtual network controller devices such as SD-WAN vSmart controller and the like. In some embodiments, the controller 350 can establish a secure connection to each device in the network 300 and distribute routing and policy information via a control plane protocol. The control plane protocol may include OMP, OSPF, IS-IS, Border Gateway Protocol, PIM, IGMP, ICMP, ARP, BFD, LACP, etc. In some embodiments, the controller 350 is capable of transmitting the cluster state 340 of the cluster 310 in the network 300 to any device in the network 300. In some embodiments, the controller 350 transmits the cluster state 340 only to the edge router 330 and the border router 320 in the cluster 310. In some embodiments, the controller 350 can operate as a route reflector. The controller 350 can also orchestrate the secure connectivity between two or more devices in the network. For example, in some embodiments, the controller 350 can distribute key information between (one or more) network devices. This allows the network to support secure network protocols or applications (e.g., IPSec, TLS, SSH, etc.) without IKE and achieves the scalability of the network.
[0048] The initial flow 360 represents any suitable flow of data packets, traffic, or messages that can be exchanged between devices of the computer network 300 from the first branch node 380 to the second branch node (e.g., server 385). The return flow 360 can be exchanged on the link using a predefined network communication protocol (e.g., TCP / IP, UDP, ATM protocol, frame relay protocol, or any other suitable protocol). In the illustrated embodiment, the initial flow 360 is exchanged on the link of devices in the network 300 using the TCP protocol that performs the first step of the TCP three-way handshake. In particular, in the illustrated embodiment, the initial flow 360 uses the TCP syn message type, which is used to initiate and establish a connection between the first branch node 380 and the server 385.
[0049] The return flow 370 represents any suitable flow of data packets, traffic, or messages that can be exchanged between devices in the computer network 300 from a branch node (e.g., server 385) to the first branch node 380 in response to receiving the initial flow 360. The return flow 370 can be exchanged on the link using a predefined network communication protocol (e.g., TCP / IP, UDP, ATM protocol, frame relay protocol, or any other suitable protocol). In the illustrated embodiment, the return flow 370 is exchanged on the link of devices in the network 300 using the TCP protocol that performs the second step of the TCP three-way handshake. In the illustrated embodiment, the return flow 370 uses the TCP syn-ack message type, which is used to initiate and establish a connection between the server 385 and the first branch node 380.
[0050] SD-WAN overlay network 390 represents any suitable software-defined network topology capable of interconnecting clusters 310. SD-WAN overlay network (sometimes also referred to as network fabric or fabric) is a virtual network topology built on top of an underlying physical network (i.e., underlying network). SD-WAN overlay network can create a logical topology for virtually connecting devices built on top of any physical network topology. SD-WAN overlay network 390 can include GRE or mGRE, MPLS VPN or VPLS, IPsec or DMVPN, CAPWAP, VXLAN, LISP, OTV and ACI TM wait.
[0051] Data center organization 395 represents any suitable network topology for implementing a data center and can be interconnected with cluster 310. Data center organization 395 may include an SDN data center organization, a traditional local area network, a spine and leaf architecture, etc. For example, data center organization 395 may be ACI TMIn some embodiments, the data center organization 395 may include an SD-WAN overlay network. In some embodiments, the data center organization 395 may be part of the SD-WAN overlay network 390.
[0052] In an exemplary embodiment of operation, the controller 350 determines the cluster state 340 of the border routers 320a, 320b, and 320c in the cluster 310. For example, in the illustrated embodiment, the cluster state 340 in the cluster 310 is three active border routers 320a, 320b, and 320c. The controller 350 transmits the cluster state 340 to the edge router 330. Similarly, the controller 350 also transmits the cluster state 340 to the border routers 320a to 320c. The edge router 330 and the border routers 320a to 320c will then generate a consistent hash table based on the cluster state 340 transmitted to them by the controller 350. Based on the above Figure 1 The process of generating consistent hash tables based on cluster state 340 is discussed in more detail in the embodiments of FIG. According to some embodiments, when an incoming packet is taken as input, each consistent hash table will route the packet to a specific owner border router 320 in cluster 310 based on consistent hashing.
[0053] In an exemplary embodiment of operation, the initial flow 360 begins when the edge router 330 receives a packet from the branch node 380. The edge router 330 uses the five-tuple of the initial flow as a key to perform a consistent hash lookup based on the consistent hash table generated by the edge router 330. The five-tuple refers to a set of five different values that comprise a TCP / IP connection, including the source IP address, source port number, destination IP address, destination port number, and the protocol in use. The edge router 330 uses the result of the consistent hash lookup to force the routing of the initial flow 360 to the border router 320b of the cluster 310. The border router 320b will then route the initial flow 160 to its destination device through the data center organization 395, which is the server 385 in the illustrated embodiment.
[0054] Still in accordance with an exemplary embodiment of operation, the return flow 370 begins when the server 385 sends a TCP syn-ack response message through the data center organization 395. However, because the server 385 is part of a different network than the SD-WAN overlay network 390, the server 385 is not aware of the consistent hash table or cluster state 340 of the cluster 310. Therefore, the server 385 can route the return flow 370 to any one of the border routers 320a to 320c in the cluster 310. But because each border router 320a to 320c knows the cluster state 340, the consistent hash table, and the return flow 370 is from a different network, when starting to process the packet response from the server 385, each border router is able to perform a consistent hash lookup and route the return flow 370 through itself if the border router is the correct flow owner border router, or the border router is able to transfer the return flow 370 to the correct flow owner border router in the cluster 310. In the illustrated embodiment, the border router 320c receives the TCP syn-ack response message from the server 385. Border router 320c uses the five-tuple of the return flow as a key to perform a consistent hash lookup based on the consistent hash table generated by border router 320c. Border router 320c uses the result of the consistent hash lookup to transfer the routing of return flow 370 to border router 320b of cluster 310. Importantly, the use of a consistent hash lookup will cause the return flow 370 to be symmetrically routed through the same border router 320 as the initial flow 360. Similarly, border router 320b uses the five-tuple of the return flow as a key to perform a consistent hash lookup based on the consistent hash table generated by border router 320b. Border router 320b uses the result of the consistent hash lookup to verify that it is the owner node, and then routes the return flow 370 to its destination device through the SD-WAN overlay network 390, which is the branch node 380 in the illustrated embodiment.
[0055] Modifications, additions, or omissions may be made to the network 300 without departing from the scope of the present disclosure. For example, any number of border routers 320 may be included in the cluster 310. As another example, any number of edge routers 330 and branch nodes 380 may be included in the network 300. In some embodiments, for traffic from non-SD-WAN data center network scenarios, the SD-WAN-only environment may be implemented as described in accordance with Figure 3 The illustrated embodiment performs symmetric forwarding with only local hashing. However, using this may result in transfers in both directions of the flow, which will be less efficient than network-wide hashing.
[0056] Figure 4An example method for stateless symmetric forwarding of packets through a cluster is shown according to some embodiments of the present disclosure.
[0057] At step 405, the method 400 starts. At step 410, the controller 150 determines the cluster state 140 of the border routers 120 in the cluster 110. In some embodiments, the controller 150 can establish a point-to-point tunnel between each border router 120 in the cluster 110. In some embodiments, in order to determine the cluster state 140 of the border routers 120 in the cluster 110, the controller 150 will run a BFD algorithm to detect the activity of all border routers 120 in the cluster 110. In some embodiments, the controller 150 can convert the data determined by running the BFD algorithm into a communicable format, such as a string or an integer. In some embodiments, the controller 150 can also transmit certain identification information about each border router 120 in the cluster 110, such as its IP address or its universally unique identifier (UUID).
[0058] At step 415, the cluster state 140 is transmitted to at least one edge router 130 in the network 100. In some embodiments, the controller 150 is capable of transmitting the cluster state 140 of the cluster 110 via an OMP. In some embodiments, the controller 150 transmits the cluster state 140 of the cluster 110 via a tunnel (e.g., an IPSec tunnel). In other embodiments, the controller 150 does not transmit the cluster state 140 at all, but rather the cluster state 140 is transmitted directly from the cluster 110 to the edge router 130 via a TCP / IP link or through an SD-WAN overlay network. In some embodiments, the cluster state 140 is transmitted to one or more branch nodes 180 in the network 100.
[0059] At step 420, one or more devices or nodes in the network 100 generate a network-level consistent hash based on the cluster state 140 of the cluster 110. In some embodiments, a consistent hash table is generated, where the five tuples of the initial flow 160 and the return flow 170 are the keys of the hash table, and each index in the consistent hash table is an active border router 120 in the cluster 110 based on the cluster state 140. In some embodiments, each edge router 130 generates its own network-level consistent hash based on the cluster state 140. In some embodiments, the controller 150 generates the network-level consistent hash and then transmits the network-level consistent hash table to each edge router 130. In some embodiments, one or more border routers 120 generate the network-level consistent hash. In some embodiments, one or more devices or nodes in the network 100 update the network-level consistent hash based on determining that the cluster state 140 of the cluster 110 has changed.
[0060] At step 425 , edge router 130 a receives a packet from branch node 180 a . The packet is part of initial flow 160 .
[0061] At step 430, edge router 130a determines whether it knows network-level consistent hashing. If edge router 130a does know network-level consistent hashing, method 400 proceeds to step 440. If edge router 130a does not know network-level consistent hashing, method 400 proceeds to step 435.
[0062] At step 435 , edge router 130 a routes initial flow 160 to cluster 110 . However, because edge router 130 a is not aware of network-level consistent hashing, edge router 130 a may route initial flow 160 to an incorrect border router 120 in cluster 110 .
[0063] At step 440, edge router 130a uses the five-tuple of the initial flow as a key to perform a consistent hash lookup according to the consistent hash table generated by edge router 130a. Edge router 130a uses the result of the consistent hash lookup to force the routing of initial flow 160 to go to border router 120a of cluster 110a. Similarly, border router 120a uses the five-tuple of the initial flow as a key to perform a consistent hash lookup according to the consistent hash table generated by border router 120a. Border router 120a uses the result of the consistent hash lookup to force the routing of initial flow 160 to go to border router 120c of cluster 110b. Border router 120c uses the five-tuple of the initial flow as a key to perform a consistent hash lookup according to the consistent hash table generated by border router 120c. A five-tuple refers to a set of five different values including a TCP / IP connection, including a source IP address, a source port number, a destination IP address, a destination port number, and a protocol in use. Border router 120c may then use the results of the consistent hash lookup to force routing of initial flow 160 to edge router 130d. Edge router 130d may then route initial flow 160 to its destination device, which will be branch node 180b.
[0064] At step 445, method 400 may end. In some embodiments, method 445 may then proceed to additional steps in accordance with the present disclosure.
[0065] Various embodiments may perform some, all or none of the above steps. For example, some embodiments may omit step 420 under certain conditions. In addition, some embodiments may perform these steps in a different order or in parallel. In addition, one or more steps may be repeated. For example, steps 425 to 440 may be repeated for any number of initial flows 160. Although discussed as edge router 130, controller 150, and border router 120 performing the steps of the method, any suitable component of computer network 100 may perform one or more steps of the method.
[0066] Figure 5 An example method for stateless symmetric forwarding of packets through a cluster connected to two different types of computer networks is shown in accordance with some embodiments of the present disclosure.
[0067] At step 505, the method 500 starts. At step 510, the controller 350 determines the cluster state 340 of the border routers 320 in the cluster 310. In some embodiments, the controller 350 can establish a point-to-point tunnel 250 between each border router 320 in the cluster 310. In some embodiments, in order to determine the cluster state 340 of the border routers 320 in the cluster 310, the controller 350 will run a BFD algorithm to detect the activity of all border routers 320 in the cluster 310. In some embodiments, the controller 350 can convert the data determined by running the BFD algorithm into a communicable format, such as a string or an integer. In some embodiments, the controller 350 can also transmit certain identification information about each border router 320 in the cluster 310, such as its IP address or its UUID.
[0068] At step 515, the cluster state 340 is transmitted to at least one edge router 330 in the network 300. In some embodiments, the controller 350 is capable of transmitting the cluster state 340 of the cluster 310 via OMP. In some embodiments, the controller 350 transmits the cluster state 340 of the cluster 310 via a tunnel (e.g., an IPSec tunnel). In other embodiments, the controller 350 does not transmit the cluster state 340 at all, but rather the cluster state 340 is transmitted directly from the cluster 310 to the edge router 330 via a TCP / IP link or through an SDWAN overlay network. In some embodiments, the cluster state 340 is transmitted to one or more branch nodes 380 in the network 300.
[0069] At step 520, one or more devices or nodes in the network 300 generate a network-level consistent hash based on the cluster state 340 of the cluster 310. In some embodiments, a consistent hash table is generated, where the five tuples of the initial flow 360 and the return flow 370 are keys of the hash table, and each index in the consistent hash table is an active border router 320 in the cluster 310 based on the cluster state 340. In some embodiments, each edge router 330 generates its own network-level consistent hash based on the cluster state 340. In some embodiments, the controller 350 generates the network-level consistent hash and then transmits the network-level consistent hash table to each edge router 330. In some embodiments, one or more border routers 320 generate the network-level consistent hash.
[0070] At step 525, edge router 330 receives a packet from branch node 380. The packet is part of initial flow 360.
[0071] At step 530, edge router 330 uses the five-tuple of the initial flow as a key to perform a consistent hash lookup according to the consistent hash table generated by edge router 330. The five-tuple refers to a set of five different values that comprise a TCP / IP connection, including the source IP address, source port number, destination IP address, destination port number, and the protocol in use. Edge router 330 uses the result of the consistent hash lookup to force the routing of initial flow 360 to edge router 320b of cluster 310. Edge router 320b will then route initial flow 160 to its destination device through data center organization 395, which will be server 385.
[0072] At step 535 , the border router 320 c receives the TCP syn-ack response message packet of the return flow 370 from the server 385 .
[0073] At step 540, border router 320c determines whether the packet from return flow 370 should be routed through border router 320c according to the network-level consistent hashing. Border router 320c uses the five-tuple of the return flow as a key to perform a consistent hash lookup according to the consistent hash table generated by border router 320c. Using the result of the consistent hash lookup, if 320c is the correct border router for return flow 370, method 500 proceeds to step 545. Using the result of the consistent hash lookup, if 320c is not the correct border router for return flow 370, method 500 proceeds to step 550.
[0074] At step 545, the border router 320c uses the five-tuple of the return flow as a key to perform a consistent hash lookup against the consistent hash table generated by the border router 320b. The border router 320c uses the result of the consistent hash lookup to verify that it is the owner node and then routes the return flow 370 to its destination device through the SD-WAN overlay network 390, which will be the branch node 380.
[0075] At step 550, border router 320c transfers the routing of return flow 370 to border router 320b of cluster 310. Importantly, the use of consistent hash lookup will result in return flow 370 being symmetrically routed through the same border routers 320 that initial flow 360 traversed.
[0076] At step 555, the border router 320b uses the five-tuple of the return flow as a key to perform a consistent hash lookup according to the consistent hash table generated by the border router 320b. The border router 320b verifies that it is the owner node using the result of the consistent hash lookup and then routes the return flow 370 to its destination device through the SD-WAN overlay network 390, which will be the branch node 380.
[0077] At step 560, method 500 may end. In some embodiments, method 500 may then proceed to additional steps in accordance with the present disclosure.
[0078] Various embodiments may perform some, all or none of the above steps. For example, some embodiments may omit step 520 under certain conditions. In addition, some embodiments may perform these steps in a different order or in parallel. In addition, one or more steps may be repeated. For example, steps 525 to 560 may be repeated for any number of initial flows 360. Although discussed as edge router 330, controller 350, and border router 320 performing the steps of the method, any suitable component of computer network 300 may perform one or more steps of the method.
[0079] In summary, a system of one embodiment provides stateless symmetric forwarding of packets in a computer network. The system includes a memory and a processor. The system is operable to determine a cluster state of a plurality of border routers in a cluster. The system is operable to transmit the cluster state to at least one branch node in the computer network. The system is operable to generate a network-level consistent hash based on the cluster state. The system is operable to route a first packet through a first border router among a plurality of border routers in the cluster using network-level consistent hashing. After the first packet is sent through the first border router, the system is also operable to route a second packet through a first border router among a plurality of border routers in the cluster using network-level consistent hashing.
[0080] Where appropriate, the computer-readable non-transitory storage medium herein may include one or more semiconductor-based circuits or other integrated circuits (ICs) (e.g., field programmable gate arrays (FPGAs) or application-specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard disk drives (HMDs), optical disks, optical disk drives (ODDs), magneto-optical disks, magneto-optical drives, floppy disks, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM drives, secure digital (SECURE DITIGAL) cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or more of these media. Where appropriate, the computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile.
[0081] As used herein, "or" is inclusive and not exclusive, unless expressly stated otherwise or the context indicates otherwise. Thus, as used herein, "A or B" means "A, B, or both," unless expressly stated otherwise or the context indicates otherwise. Additionally, "and" means both jointly and severally, unless expressly stated otherwise or the context indicates otherwise. Thus, as used herein, "A and B" means "A and B, jointly or severally," unless expressly stated otherwise or the context indicates otherwise.
[0082] The scope of the present disclosure covers all changes, substitutions, variations, alterations and modifications to the example embodiments described or illustrated herein that will be understood by those of ordinary skill in the art. The scope of the present disclosure is not limited to the example embodiments described or illustrated herein. In addition, although the present disclosure describes and illustrates each embodiment herein as including specific components, elements, features, functions, operations or steps, any one of these embodiments may include any combination or arrangement of any components, elements, features, functions, operations or steps described or illustrated anywhere in this article that will be understood by those of ordinary skill in the art. In addition, the reference to the device, or system, or the component of the device or system being adapted, arranged, capable, configured, enabled, operable or operated to perform a specific function in the attached claims covers the device, system, component (regardless of whether the device, system, component or the specific function is activated) being opened or unlocked, as long as the device, system or component is so adapted, arranged, capable, configured, enabled, operable or operated. In addition, although the present disclosure describes or illustrates a specific embodiment as providing a specific advantage, a specific embodiment may not provide these advantages, provide some advantages or provide all of these advantages.
Claims
1. A system comprising: one or more processors; as well as One or more computer-readable non-transitory storage media comprising instructions that, when executed by the one or more processors, cause one or more components of the system to perform operations comprising: determining a cluster state of a plurality of border routers in the cluster; transmitting the cluster state to at least one branch node; Generating a network-level consistent hash based on the cluster state; and A first packet is routed through a first border router of the plurality of border routers in the cluster using the network-level consistent hashing.
2. The system according to claim 1, wherein: Determining a cluster state of a plurality of border routers in a cluster includes implementing a bidirectional forwarding detection protocol.
3. The system according to claim 1 or 2, wherein: Transmitting the cluster state to at least one branch node is performed by a controller.
4. The system according to any one of claims 1 to 3, wherein the operation further comprises: A second packet is routed through the first of the plurality of border routers in the cluster using the network-level consistent hashing.
5. The system of claim 4, wherein the operations further comprise: The second packet received by the second border router in the cluster is transferred using the network-level consistent hashing by routing the second packet from the second border router in the cluster to the first border router in the cluster.
6. The system according to any one of claims 1 to 5, wherein: Generating a network-level consistent hash based on the cluster state includes: generating a consistent hash table, wherein a key of the consistent hash table includes the five-tuple of the first group.
7. The system according to any one of claims 1 to 6, wherein the operations further comprise: The network-level consistent hashing is updated based on a determination that a cluster state of the cluster has changed.
8. A method comprising: determining a cluster state of a plurality of border routers in the cluster; transmitting the cluster state to at least one branch node; Generating a network-level consistent hash based on the cluster state; as well as A first packet is routed through a first border router of the plurality of border routers in the cluster using the network-level consistent hashing.
9. The method according to claim 8, wherein: Determining a cluster state of a plurality of border routers in a cluster includes implementing a bidirectional forwarding detection protocol.
10. The method according to claim 8 or 9, wherein: Transmitting the cluster state to at least one branch node is performed by a controller.
11. The method according to any one of claims 8 to 10, further comprising: A second packet is routed through the first of the plurality of border routers in the cluster using the network-level consistent hashing.
12. The method according to claim 11, further comprising: The second packet received by the second border router in the cluster is transferred using the network-level consistent hashing by routing the second packet from the second border router in the cluster to the first border router in the cluster.
13. The method according to any one of claims 8 to 12, wherein: Generating a network-level consistent hash based on the cluster state includes: generating a consistent hash table, wherein a key of the consistent hash table includes the five-tuple of the first group.
14. The method according to any one of claims 8 to 13, further comprising: The network-level consistent hashing is updated based on a determination that a cluster state of the cluster has changed.
15. One or more computer-readable non-transitory storage media containing instructions that, when executed by a processor, cause the processor to perform operations comprising: determining a cluster state of a plurality of border routers in the cluster; transmitting the cluster state to at least one branch node; Generating a network-level consistent hash based on the cluster state; as well as A first packet is routed through a first border router of the plurality of border routers in the cluster using the network-level consistent hashing.
16. The one or more computer-readable non-transitory storage media of claim 15, wherein: Determining a cluster state of a plurality of border routers in a cluster includes implementing a bidirectional forwarding detection protocol.
17. One or more computer-readable non-transitory storage media according to claim 15 or 16, wherein: Transmitting the cluster state to at least one branch node is performed by a controller.
18. The one or more computer-readable non-transitory storage media of any one of claims 15 to 17, the operations further comprising: A second packet is routed through the first of the plurality of border routers in the cluster using the network-level consistent hashing.
19. The one or more computer-readable non-transitory storage media of claim 18, the operations further comprising: The second packet received by the second border router in the cluster is transferred using the network-level consistent hashing by routing the second packet from the second border router in the cluster to the first border router in the cluster.
20. The one or more computer-readable non-transitory storage media of any one of claims 15 to 19, the operations further comprising: The network-level consistent hashing is updated based on a determination that a cluster state of the cluster has changed.
21. An apparatus comprising: means for determining a cluster state of a plurality of border routers in the cluster; means for transmitting said cluster state to at least one branch node; means for generating a network-level consistent hash based on said cluster state; as well as Means for routing a first packet through a first border router of the plurality of border routers in the cluster using the network-level consistent hashing.
22. The apparatus according to claim 21, further comprising: An apparatus for implementing the method according to any one of claims 9 to 14.
23. A computer program, a computer program product or a computer readable medium comprising instructions which, when executed by a computer, cause the computer to perform the steps of the method according to any one of claims 8 to 14.
