Network device with smooth failover of sessions

Abstract

The invention relates to a network device with smooth failover of a session. Techniques are disclosed for a network device that performs a gradual failover of a session from a first link to a second link. For example, a network device forwards network traffic for a plurality of sessions via a first link. The network device determines that the performance of the first link does not meet the performance requirement. Based on the determination, the network device forwards the network traffic of the first portion of the plurality of sessions via the second link rather than the first link. The network device determines that the performance of the second link satisfies the performance requirement while bearing the network traffic of the first portion. Based on the determination, the network device forwards a second, larger portion of the network traffic of the plurality of sessions via the second link rather than the first link.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit of U.S. Patent Application No. 18 / 933,671, filed October 31, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates generally to computer networks, and more specifically to routing packets within computer networks. Background Technology

[0004] A computer network is a collection of interconnected computing devices that can exchange data and share resources. Example computing devices include routers, switches, and other Layer 2 (L2) network devices operating within Layer 2 (i.e., the data link layer) of the Open Systems Interconnection (OSI) reference model, and Layer 3 (L3) network devices operating within Layer 3 (i.e., the network layer) of the OSI reference model. Network devices within a computer network typically include control units that provide control plane functions for the network device and forwarding components for routing or switching data units.

[0005] Computing devices can establish “network sessions” (also referred to herein as “sessions”) to enable communication between devices on a computer network. A session can be bidirectional because it comprises packets traveling in both directions between a first device and a second device. For example, a session includes a forward packet flow originating from the first device and destined for the second device, and a reverse packet flow originating from the second device and destined for the first device. The forward and reverse packet flows of a session are related because the source address and source port of the forward packet flow are the same as the destination address and destination port of the reverse packet flow, and the destination address and destination port of the forward packet flow are the same as the source address and source port of the reverse packet flow. To establish a session, a computing device can use one or more communication session protocols, including Transmission Control Protocol (TCP), Transport Layer Security (TLS), User Datagram Protocol (UDP), Internet Control Message Protocol (ICMP), etc. Summary of the Invention

[0006] Typically, this disclosure describes techniques for performing a “smooth” or gradual failover of network traffic associated with a session and forwarded by a network device via a first link to a second link, such that the network device iteratively migrates network traffic to increase the amount of sessions forwarded from the first link to the second link. For example, a network device as described herein forwards network traffic for multiple sessions via the first link. The network device determines that the performance of the first link does not meet performance requirements. In some examples, the performance requirements are specified by a service level agreement (“SLA”) for the session. In some examples, the performance requirements are the performance requirements of an application or service associated with the multiple sessions. Based at least in part on this determination, the network device forwards a first portion of the network traffic for the multiple sessions via the second link instead of the first link.

[0007] A network device determines that the performance of a second link meets a performance requirement when carrying a first portion of network traffic for multiple sessions. Based on this determination, the network device forwards a second, larger portion of the network traffic for the multiple sessions via the second link instead of the first link. The network device continues to iteratively transfer a larger number of sessions from forwarding via the first link to forwarding via the second link, and tests the second link to ensure compliance with the performance requirement, until the entire session is forwarded via the second link. In some examples, the network device may increase the proportion of sessions whose network traffic is forwarded via the second link according to linear, logarithmic, geometric, exponential, or other algorithms. Therefore, using the techniques of this disclosure, when the first link fails to meet the performance requirement, the network device described herein can iteratively migrate network traffic from the first link to the second link for an increasing number of sessions, so as to progressively test the second link's ability to meet the performance requirement as the amount of network traffic increases, without overwhelming the second link with the migrated network traffic.

[0008] The techniques disclosed herein can provide specific improvements to computer-related fields of computer networks with one or more practical applications. For example, the techniques disclosed herein can enhance the ability of network devices to ensure that network traffic associated with a session is forwarded via a link that meets the performance requirements of the corresponding session. More specifically, the techniques disclosed herein enable network devices, as described herein, to progressively test the ability of a second link with small amounts of network traffic before moving the entire network traffic from the first link in the event of a first link failure. This provides several advantages, such as ensuring that the second link meets performance requirements with progressively larger amounts of network traffic. Furthermore, migrating a subset of network traffic from the first link reduces overuse of the first link, allowing the first link to recover and meet performance requirements. Moreover, in conventional systems that do not use the techniques disclosed herein, if a network device migrates the entire network traffic forwarded via the first link to the second link, the second link can become overused, causing it to fail to meet performance requirements, resulting in "oscillations" in which the network device migrates network traffic from the first link to the second link, back to the first link, and back to the second link again. Using the techniques disclosed herein, network devices can migrate only the amount of network traffic that the second link can handle, while still meeting performance requirements, in order to avoid overuse of the second link and mitigate the occurrence of "oscillations".

[0009] In one example, this disclosure describes a network apparatus including: processing circuitry communicating with a storage medium, the processing circuitry being configured to: forward a first portion of network traffic of the multiple sessions via a second link instead of the first link, based at least in part on determining that the performance of forwarding network traffic of the multiple sessions via a first link does not meet performance requirements; and forward a second portion of network traffic of the multiple sessions via the second link instead of the first link, based at least in part on determining that the performance of forwarding the first portion of network traffic of the multiple sessions via the second link meets performance requirements, wherein the second portion is larger than the first portion.

[0010] In another example, this disclosure describes a method comprising: forwarding a first portion of network traffic of multiple sessions via a second link instead of the first link, based at least in part on the determination that the performance of forwarding network traffic of multiple sessions via a first link does not meet performance requirements; and forwarding a second portion of network traffic of multiple sessions via the second link instead of the first link, based at least in part on the determination that the performance of forwarding network traffic of the first portion of the multiple sessions via the second link meets performance requirements, wherein the second portion is larger than the first portion.

[0011] In another example, this disclosure describes a non-transitory computer-readable medium including instructions that, when executed, are configured to cause processing circuitry of a network device to: forward a first portion of network traffic of multiple sessions via a second link instead of the first link, based at least in part on the determination that the performance of forwarding network traffic of the first portion of multiple sessions via a first link does not meet performance requirements; and forward a second portion of network traffic of the multiple sessions via the second link instead of the first link, based at least in part on the determination that the performance of forwarding network traffic of the first portion of multiple sessions via the second link meets performance requirements, wherein the second portion is larger than the first portion.

[0012] Details of one or more examples of the technology disclosed herein are illustrated in the following figures and description. Other features, objects, and advantages of these technologies will be apparent from the specification, figures, and claims. Attached Figure Description

[0013] Figure 1 This is a block diagram illustrating an example computer network system according to the technology of this disclosure.

[0014] Figure 2 This is a block diagram illustrating an example network device according to the technology of this disclosure.

[0015] Figure 3 This is a flowchart illustrating an example operation of the technology according to this disclosure.

[0016] Throughout the accompanying drawings and description, the same reference characters refer to the same elements. Detailed Implementation

[0017] Some network devices, such as Session Intelligent Routers (SSRs) provided by Juniper Networks, forward network traffic on a per-session basis. A session is a bidirectional flow of network traffic between two client devices. If the first link along the network path between the two client devices fails (or its performance deteriorates to the point that it no longer meets performance requirements), a conventional network device can migrate the forwarding of network traffic for all corresponding sessions forwarded via the first link to a second link. However, the second link may not have sufficient resources to handle all the migrated session network traffic, except for any pre-existing traffic forwarded via it. Therefore, the overall migration of network traffic from the first link to the second link can cause the second link to fail (or suffer performance degradation to the point that it no longer meets performance requirements) due to the overwhelming amount of traffic. In some cases, this can cause "oscillations" when a network device migrates traffic from the use of the first link to the use of the second link and then back again. For example, transmitting traffic for all sessions can overwhelm the resources of the second link, causing it to deteriorate or fail to meet performance requirements. Meanwhile, the reduction in network traffic forwarded via the first link can cause the first link to recover. Subsequently, the network device can migrate traffic back to the first link, causing the first link to fail and the second link to recover for the same reason. The network device can then migrate traffic back to the second link again, and so on, causing oscillations between the first and second links because each link cannot meet the network traffic demands of all migration sessions, while links without network traffic recover due to reduced network resources.

[0018] According to the technology disclosed herein, when a failure is determined in the first link (or its performance has degraded below one or more performance requirements of the session), the network apparatus described herein can migrate a progressively increasing portion of the session's network traffic to a second link to ensure that the second link maintains compliance with performance requirements through the additional resource consumption of each portion of the migrated network traffic. In some examples, the performance requirements are specified by the session's Service Level Agreement (“SLA”). In some examples, the performance requirements are the performance requirements of an application or service associated with multiple sessions. The network apparatus migrates increasing amounts of session network traffic to the second link while monitoring compliance with performance requirements, until all session network traffic has been migrated from the first link to the second link.

[0019] In some examples, after migrating network traffic for a first portion of a session, rather than all of the session, from the first link to the second link, it can be determined that both the first and second links meet performance requirements. For example, migrating network traffic for the first portion of the session may be sufficient to alleviate network congestion on the first link, thereby improving the performance of the first link to a level sufficient to meet performance requirements. In this case, the network may (1) maintain the current distribution of session traffic between the first and second links; (2) continue migrating another portion (or all) of the session's network traffic from the first link to the second link; or (3) migrate another portion (or all) of the session's network traffic from the second link back to the first link. In some examples, this decision may be a configuration option of the network device configured by an administrator or user. Thus, by transferring network traffic used for the session to the new link at an increasing rate, while monitoring performance requirements compliance of the new link each time session network traffic is migrated, the network device described herein can ensure that the new link meets performance requirements with increasing traffic. Therefore, the techniques described herein improve the ability of a network device to ensure that links selected for forwarding network traffic meet performance requirements, and improve network throughput by reducing congestion on individual links.

[0020] In some examples, the network device applies a logarithmic scaling algorithm to determine the stepwise number of sessions for which network traffic is migrated from a first link to a second link, migrating network traffic for, for example, 1 session, then 10 sessions, then 100 sessions, 1000 sessions, and so on. After migrating network traffic for each "step" or a portion of a session, the network device monitors the second link's compliance with performance requirements to ensure that the second link can support the increased traffic levels without violating performance requirements. In some examples, the network device may apply different algorithms to determine the stepwise session volume, including but not limited to linear, polynomial, exponential, or factorial algorithms.

[0021] Figure 1 This is a block diagram illustrating an example computer network system 2 according to the technology of this disclosure. Figure 1In the example, computer network system 2 includes service provider networks 150A-150D (collectively, "service provider network 150") configured to provide wide area network (WAN) connections to different customer networks 140A-140B (collectively, "customer network 140"). Network devices 110A-110I (collectively, "network devices 110") of service provider network 150 provide access to service provider network 150 for client devices 100A-100B (collectively, "client devices 100") associated with customer network 140. In some examples, customer network 140 is an enterprise network. For ease of illustration, customer network 140A is described as having a single client device 100A, and customer network 140B is described as having a single client device 100B, but each of customer networks 140 may include any number of client devices. Typically, customer network 140 includes a plurality of client devices 100, each of which can communicate with each other via service provider network 150, as described in more detail below. Communication links 16A-16J (collectively referred to as link "16") can be Ethernet, ATM, or any other suitable network connection.

[0022] Network device 110 can be implemented using any network device, such as a switch, router, gateway, or other suitable network device capable of sending and receiving network traffic. Customer network 140 can be a network for geographically separated sites, such as those of a business. Each of customer network 140 may include additional customer equipment, such as one or more non-edge switches, routers, hubs, gateways, security devices (such as firewalls, intrusion detection and / or intrusion prevention devices), servers, computer terminals, laptops, printers, databases, wireless mobile devices (such as cellular phones or personal digital assistants), wireless access points, bridges, cable modems, application accelerators, or... Figure 1 Other routers not described in the text. Figure 1 The configuration of computer network system 2 shown is merely an example. For instance, computer network system 2 may include any number of client networks 140. Nevertheless, for ease of description, Figure 1 Only customer networks 140A-140B are shown in the image.

[0023] Service provider network 150 refers to one or more publicly accessible computer networks owned and operated by one or more service providers. While computer network system 2 is in Figure 1The example shown comprises multiple interconnected service provider networks 150; however, in other examples, the computer network system 2 may alternatively comprise a single service provider network providing connectivity between customer networks 140. Service providers are typically large telecommunications entities or companies. Each service provider network 150 is typically a large L3 computer network. Each service provider network 150 is an L3 network, in that sense, inherently supporting L3 operations as described in the OSI model. Common L3 operations include those performed according to L3 protocols such as the Internet Protocol (IP). L3 is also referred to as the "network layer" in the OSI model, and throughout this disclosure, the term L3 may be used interchangeably with the phrase "network layer."

[0024] Although not shown, each service provider network 150 may be coupled to one or more networks managed by other providers, and thus may form part of a large-scale public network infrastructure (e.g., the Internet). Therefore, customer network 140 can be considered as an edge network of the Internet. Each service provider network 150 may provide internet access to computing devices (such as client devices 100) within customer network 140, and may allow computing devices within customer network 140 to communicate with each other.

[0025] Although additional network devices are not shown for ease of explanation, it should be understood that System 2 may include additional network and / or computing devices, such as, for example, one or more additional switches, routers, hubs, gateways, security devices (such as firewalls, intrusion detection and / or intrusion prevention devices), servers, computer terminals, laptop computers, printers, databases, wireless mobile devices (such as cellular phones or personal digital assistants), wireless access points, bridges, cable modems, application accelerators, or other routers. Furthermore, although the elements of System 2 are illustrated as directly coupled, it should be understood that one or more additional network elements may be included along any of the network links 16 such that the network elements of System 2 are not directly coupled.

[0026] Each service provider network 150 typically provides multiple residential and commercial services to the customer network 140, including residential and commercial data services (which are often referred to as “Internet services” because these data services allow access to a collection of publicly accessible networks known as the Internet), residential and commercial telephone and / or voice services, and residential and commercial television services.

[0027] In some examples, network device 110 may implement a stateful, session-based routing scheme that allows each network device 110 to perform path selection and traffic engineering independently. The use of session-based routing allows network device 110 to avoid using a centralized controller, such as a software-defined networking (SDN) controller, to perform path selection and traffic engineering. In this way, network device 110 may be more efficient and scalable for large networks where the use of an SDN controller would be impractical. Furthermore, the use of session-based routing allows network device 110 to avoid the use of tunnels, thereby saving considerable network resources by avoiding the need to perform encapsulation and decapsulation at tunnel endpoints. In some examples, network device 110 implements session-based routing as Secure Vector Routing (SVR) provided by Juniper Networks.

[0028] exist Figure 1 In the example, client device 100A of system 2 establishes session 40 with client device 100B. Network device 110 facilitates the establishment of session 40 by transmitting network traffic between client device 100A and client device 100B. In some examples, client device 100A can be considered a "source" device because client device 100A initiates session 40 between client device 100A and client device 100B; for example, client device 100A is the "source" of packets in the forward flow of the session. Session 40 includes a forward packet flow originating from client device 100A and destined for client device 100B, and a reverse packet flow originating from client device 100B and destined for client device 100A. The forward flow of session 40 traverses a first path, including, for example, client device 100A, network devices 110A-110I, and client device 100B. As described in more detail below, network device 110 enables the exchange of traffic between customer network 140A and customer network 140B via service provider network 150.

[0029] Client device 100A can establish session 40 with client device 100B according to one or more L2 or L3 communication session protocols including Ethernet, TCP, or UDP. As described in more detail below, client network 140A can form a first network and client network 140B can form a second network. Network device 110 operates to extend client network 140A across service provider network 150 to client network 140B. In this way, client network 140A and client network 140B can operate as if they were part of the same network, even if client network 140A and client network 140B can be logically isolated from each other and geographically separated. Furthermore, network device 110 is operable such that the presence of service provider network 150 between client network 140A and client network 140B is transparent to client device 100.

[0030] In some examples, network device 110 may extend session 40 across service provider network 150 according to one or more communication session protocols, including TCP or UDP. For example, to establish session 40 according to TCP so that data can be exchanged according to TCP, network devices 110A and 110B perform a three-way handshake. Network device 110A sends a first data packet including a “SYN” flag to network device 110B. Network device 110B acknowledges receipt of the first data packet by responding to network device 110A with a second data packet including a “SYN-ACK” flag. Network device 110A acknowledges receipt of the second data packet by responding to network device 110B with a third data packet including an “ACK” flag. After sending the third data packet, session 40 is established according to TCP, and network devices 110A and 110B can exchange data with each other via session 40 (e.g., by exchanging data packets between client devices 100A and 100B). Additional exemplary information regarding TCP is described in the Request for Comments (RFC) 793, Internet Engineering Task Force (IETF), September 1981, available at https: / / tools.ietf.org / html / rfc793, the entire contents of which are incorporated herein by reference.

[0031] UDP is a connectionless protocol because network device 110A does not verify that network device 110B can receive data before sending it. To establish session 40 according to UDP, network device 110A sends a first data packet to network device 110B. Once network device 110A receives any data packet from network device 110B, session 40 can be considered "established" according to UDP, meaning that network device 110B has successfully received the first data packet from network device 110A, responded, and network device 110A is able to receive a response from network device 110B. Additional exemplary information about UDP is available in "User Datagram Protocol," RFC 768, IETF, August 28, 1980, at https: / / tools.ietf.org / html / rfc768, the entire contents of which are incorporated herein by reference.

[0032] exist Figure 1 In one example, when network device 110A receives a packet for a forward packet flow originating from client device 100A and destined for client device 100B, network device 110A determines whether the packet belongs to a new session (e.g., whether it is the "first" or "leading" packet of session 40). In some examples, network device 110A determines whether the source address, source port, destination address, destination port, and protocol of the first packet match entries in the session table.

[0033] If no such entry exists, network device 110A determines that the packet belongs to a new session and creates an entry in the session table. Furthermore, if the packet belongs to a new session, network device 110A can generate a session identifier for session 40. The session identifier may include, for example, the source address and source port of client device 100A, the destination address and destination port of client device 100B, and the protocol used by the first packet. Network device 110A can use the session identifier to identify subsequent packets as belonging to the same session.

[0034] In some examples, network device 110 performs stateful routing for session 40. For example, network device 110 may forward each packet of the forwarding packet flow of session 40 sequentially and along the same forwarding network path. As described herein, the "same" forwarding path may mean the same network device 110 forming a segment or at least a portion between the device initiating the packet and the device to which the packet is destined (and not necessarily the entire network path between the device initiating the packet and the device to which the packet is destined). Furthermore, network device 110 forwards each packet of the return flow of session 40 sequentially and along the same return network path. The forward network path of the forwarding packet flow of session 40 and the return network path of the return packet flow of session 40 may be the same path or different paths. By ensuring that each packet of the flow is forwarded sequentially and along the same path, network device 110 maintains the state of the entire flow at each network device 110, thereby enabling the use of stateful packet services, such as Deep Packet Inspection (DPI) or stateful firewall services.

[0035] exist Figure 1 In this example, a stateful routing session can be established from ingress network device 110A through intermediate network devices 110B-110H to egress network device 110I. In this example, network device 110A determines that the first packet is an unmodified packet of the new session 40 and is the first packet. Network device 110A modifies the first packet to include metadata specifying a session identifier (e.g., original source address, source port, destination address, and destination port). Network device 110A replaces the header of the modified first packet to specify the source address as the address of network device 110A, the source port as the port through which network device 110A forwards the modified first packet to client device 100B, the destination address as the address of the next hop for network device 110A to forward the first packet (e.g., the address of network device 110B), and the destination port as the port of the next hop for network device 110A to forward the first packet (e.g., the port of network device 110B).

[0036] Network device 110A can further identify the network services associated with session 40. For example, network device 110A can compare one or more of the source address, source port, destination address, and destination port of the session with a table of service address and port information to identify the service associated with the session. Examples of network services include Hypertext Transfer Protocol (HTTP), firewall services, proxy services, packet monitoring or measurement services, etc. For example, network device 110A can determine that the forward packet flow of session 40 specifies the destination address and destination port assigned to client device 100B. Network device 110A can then store the association between session 40 and the identified network services. As another example, if the source port and / or destination port of session 40 is 80, then network device 110A can determine that session 40 is associated with the HTTP service. In other examples, network device 110A can determine that one or more of the source address, source port, destination address, and destination port of session 40 belong to an address or port block indicating that a specific service is associated with session 40.

[0037] In some examples, network device 110A uses the determined network service for session 40 to select a forwarding path for forwarding the first packet and each subsequent packet of the forward packet flow of session 40 to client device 100B. In this way, network device 110A can perform service-specific path selection to choose the network path best suited to the service requirements. In contrast to network topologies that use an SDN controller to perform path selection, each network device 110 performs path selection. Furthermore, the use of session-based routing allows each network device 110 to make routing decisions at either the service or application level, compared to conventional routers that can only make routing decisions at the flow level.

[0038] Network device 110A forwards the modified first packet to network device 110B. Furthermore, network device 110A stores the session identifier of session 40, enabling it to identify subsequent packets from session 40 as belonging to the same session 40 and forward them along the same path as the first packet.

[0039] Intermediate network device 110B receives the modified first packet and determines whether the modified first packet includes metadata of the specified session identifier. In response to determining that the modified first packet includes metadata of the specified session identifier, intermediate network device 110B determines that network device 110B is not an ingress device, such that network device 110B does not attach the metadata of the specified session identifier.

[0040] As described above with respect to network device 110A, network device 110B determines whether a packet belongs to a new session (e.g., is the “first” or “leading” packet of the session) by determining whether the source address, source port, destination address, destination port, and protocol of the first packet match an entry in the session table. If no such entry exists, network device 110B determines that the packet belongs to a new session and creates an entry in the session table. Furthermore, if the packet belongs to a new session, network device 110B generates a session identifier for the session. The session identifier used by network device 110B to identify the first packet may differ from the session identifier used by network device 110A to identify the same session, because each network device 110A, 110B uses the header source address, source port, destination address, and destination port of the first packet to generate the session identifier, and this header information may be modified by each previous network device 110 as each network device 110 forwards the first packet along the forwarding path. In addition, each network device 110 can store the header information to identify the previous network device 110 (or "waypoint") and the next network device 110 (or "waypoint"), so that each network device 110 can reconstruct the same forward and reverse paths for each subsequent packet of the session.

[0041] Network device 110B replaces the header of the modified first packet to specify the source address (as the address of network device 110B), the source port (as the port through which network device 110B forwards the modified first packet to client device 100B), the destination address (e.g., the address of network device 110C for session 40 along the first path), and the destination port (e.g., the port of network device 110C), which is the address of the next hop for network device 110B to forward the first packet. Network device 110B forwards the modified first packet to network device 110C. Furthermore, network device 110B stores a session identifier, enabling it to identify subsequent packets belonging to the same session upon receiving subsequent packets from the same session and forward them along the same path as the first packet.

[0042] Subsequent intermediate network devices 110C-110H process the modified first packet in a manner similar to network devices 110A and 110B, such that network device 110 forwards subsequent packets of the session along the same path as the first packet. Furthermore, each network device 110 stores a session identifier for that session, which may include the identifier of a previous network device 110 along that network path. Therefore, each network device 110 can use the session identifier to forward packets of the reverse packet flow for the session back to client device 100 along the same network path.

[0043] Network device 110, which forwards packets of the forward packet flow of a session to the destination of the packet flow, is an egress or “terminal” network device. In the aforementioned example, network device 110I is a terminal network device because network device 110I can forward packets to client device 100B. Network device 110I receives a modified first packet containing metadata specifying a session identifier (e.g., original source address, source port, destination address, and destination port). Network device 110I identifies the modified first packet as a destination terminating at network device 110I by determining that the destination source address and destination source port specified in the metadata of the modified first packet correspond to a destination reachable by network device 110I (e.g., client device 100B). Network device 110I recovers the original first packet by removing metadata from the modified first packet and modifying the header of the first packet with the metadata to specify the original source address, source port, destination address, and destination port. Network device 110I forwards the recovered first packet to client device 100B. The use of session-based routing can thus form a series of waypoints (e.g., network device 110) interconnected by path “segments” (e.g., end-to-end route vectors between each waypoint).

[0044] Additional information regarding session-based routing and SVR can be found in U.S. Patent No. 9,729,439, entitled "Computer Network Packet Flow Controller," published August 8, 2017; U.S. Patent No. 9,729,682, entitled "Network Device and Method for Processing a Session Using a Packet Signature," published August 8, 2017; U.S. Patent No. 9,762,485, entitled "Network Packet Flow Controller with Extended Session Management," published September 12, 2017; and U.S. Patent No. 9,762,485, entitled "Router with Optimized Statistical Functions." U.S. Patent No. 9,871,748, entitled "FUNCTIONALITY", published January 16, 2018; U.S. Patent No. 9,985,883, entitled "NAME-BASED ROUTING SYSTEM AND METHOD", published May 29, 2018; U.S. Patent No. 10,200,264, entitled "LINKSTATUS MONITORING BASED ON PACKET LOSS DETECTION", published February 5, 2019; U.S. Patent No. 10,277,506, entitled "STATEFUL LOAD BALANCING IN A STATELESS NETWORK", published April 30, 2019; and U.S. Patent No. 10,277,506, entitled "NETWORK PACKET FLOW CONTROLLER WITHEXTENDED SESSION", published April 30, 2019. The entire contents of each of these applications are incorporated herein by reference in U.S. Patent No. 10,432,522, published October 1, 2019, entitled “Management”; and in U.S. Patent Application Publication No. 2020 / 0403890, published December 24, 2020, entitled “In-line Performance Monitoring”.

[0045] In some examples, to implement session-based routing, each network device 110 maintains a local repository of service and topology state information for each other network device 110. The service and topology state information includes services reachable from each network device 110, and the network topology used to reach these services from each network device. Each network device 110 may transmit changes to services reachable from it and / or changes to the network topology used to reach these services to a central repository, such as a server. Furthermore, each network device 110 may receive service and topology state information from each other network device 110 in system 2 from the central repository.

[0046] In the aforementioned example, network device 110A receives a packet, identifies session 40 of the forward packet flow including the packet, determines the service associated with session 40, and selects a network path for forwarding the packet. Network device 110A can use a local copy of its service and topology state information of each network device 110 to select the network path for forwarding the packet. For example, network device 110A can use the identified service associated with the packet and the network topology for reaching the identified service to select a network path that meets the SLA requirements of the service or other session performance requirements. Network device 110A can then forward packets and subsequent packets of the forward packet flow of session 40 along the selected path. In this way, network device 110A can perform service-specific path selection because network device 110 can use criteria specific to the service associated with the packet to select the network path that best suits the service requirements.

[0047] In some examples, the interface of network device 110 can be assigned to one or more "neighborhoods". A "neighborhood" is defined as a label applied to the interface of network device 110. Network devices 110 within the same neighborhood can form peer-to-peer relationships with each other. For example, each network device 110 with an interface to which a neighborhood label is applied can reach every other network device 110 with an interface to which the same neighborhood label is applied via a Layer 3 network. In some examples, one or more neighborhoods can be grouped into "regions". A region is a logical grouping of one or more neighborhoods. Typically, an Autonomous System (AS) (also referred to herein as an "authority") can be divided into one or more regions, each region comprising one or more neighborhoods.

[0048] In some examples, each network device 110 maintains a local repository of service and topology state information only for those other network devices 110 within the same neighborhood. In other examples, each network device 110 maintains a local repository of service and topology state information only for those other network devices 110 within the same neighborhood region. As an example, each service provider network 150 can be considered a distinct “region,” where each subdomain within each service provider network 150 can be considered a neighborhood within that region. In this example, each network device 110A and 110B within service provider network 150A may maintain service and topology state information only for each other, rather than for network devices 110C through 110I. Similarly, each network device 110D and 110C within service provider network 150B may maintain service and topology state information only for each other, rather than for network devices 110A-110B or 110E-110I. In other examples, the administrator may assign one or more service provider networks 150 to one or more regions, one or more neighborhoods, or a combination of regions and neighborhoods as needed by network system 2.

[0049] Additional information regarding the exchange of service and topology state information is described in the following: U.S. Patent Application Publication No. US2020 / 0366590A1, entitled "CENTRAL AUTHORITY FOR SERVICE AND TOPOLOGY EXCHANGE," published November 19, 2020; U.S. Patent Application Publication No. 2020 / 0366599, entitled "SOURCE-BASED ROUTING," published November 19, 2020; U.S. Patent Application Publication No. 2020 / 0366598, entitled "SERVICE AND TOPOLOGY EXCHANGE PROTOCOL," published November 19, 2020; and U.S. Patent Application Publication No. 2020 / 0366598, entitled "ROUTING USING SEGMENT-BASED," published November 19, 2020. U.S. Patent Application Publication No. 2020 / 0366589 entitled “METRICS”; and U.S. Patent Application No. 16 / 050,722 entitled “NETWORK NEIGHBORHOODS FOR ESTABLISHING COMMUNICATIONRELATIONSHIPS BETWEEN COMMUNICATION INTERFACES IN AN ADMINISTRATIVE DOMAIN”, filed on July 31, 2018, the entire contents of which are incorporated herein by reference.

[0050] Bidirectional Forwarding Detection (BFD) is a network protocol used to detect faults in a bidirectional path (e.g., link 16 between network devices 110A and 110B) between two network devices. BFD provides low-overhead, short-duration detection of link faults between two network devices. Furthermore, BFD provides a single mechanism for real-time detection that can be used on any medium and at any protocol layer, with broad detection time and overhead to avoid the proliferation of different methods between adjacent devices. BFD operates on top of any data protocol (network layer, link layer, tunneling, etc.) forwarded between two network devices. Typically, BFD operates in unicast, point-to-point mode. BFD packets are carried as payloads of any encapsulation protocol suitable for the medium and network.

[0051] According to BFD, network devices 110A and 110B establish a session via link 16. Typically, network devices 110A and 110B establish and terminate a BFD session via a three-way handshake. Typically, network devices 110A and 110B can declare link 16 operational only after bidirectional communication has been established between network devices 110A and 110B. However, this does not preclude the use of unidirectional links. For example, link 16 may represent a first unidirectional link from network device 110A to network device 110B, and a second unidirectional link from network device 110B to network device 110A.

[0052] Once a BFD session is established, network devices 110A and 110B periodically transmit BFD packets via link 16. Each network device 110A, 110B estimates how quickly it can send and receive BFD packets in order to negotiate with its peer network devices 110A, 110B how quickly fault detection can occur. In some examples, network devices 110A and 110B may modify these estimates in real time to accommodate network congestion, latency or bandwidth variations, or other unusual conditions. This allows for the use of a shared medium between fast and slow network devices, allowing fast network devices to detect faults more quickly while allowing slow network devices to participate in fault detection.

[0053] BFD has two operating modes: asynchronous mode and demand mode. In asynchronous mode, if one of network devices 110A and 110B stops receiving BFD packets for a certain amount of time (its length is negotiated as described above), then network devices 110A and 110B can assume that link 16 (or a component, device, or the path forming link 16) has failed. In demand mode, network devices 110A and 110B can negotiate not to send periodic BFD packets to reduce overhead. This assumes that network devices 110A and 110B have another way to verify their connectivity with each other, such as via the physical layer. However, either network device 110A or 110B can still send BFD packets if needed.

[0054] In addition, either network device 110A or 110B can use the Echo function. When this function is activated, network device 110A, for example, sends a stream of echo packets to network device 110B. Network device 110B responds by sending the echo packets back to network device 110A via its forwarding plane. Network device 110A can use the echo function to test the forwarding path of network device 110B, and vice versa. Additional example information about BFD is described in “Bidirectional Forwarding Detection (BFD),” RFC 5880, IETF, June 2010, available at https: / / datatracker.ietf.org / doc / html / rfc5880, the entire contents of which are incorporated herein by reference.

[0055] Network device 110 creates separate BFD sessions for each communication path and data protocol used between the two network devices. For example, in order to perform fault detection along the entire path between network devices 110A and 110C, different BFD sessions can be established along each link 16, such as a first BFD session along link 16 between network devices 110A and 110B and a second BFD session along link 16 between network devices 110B and 110C.

[0056] In some examples, using dedicated BFD sessions between two network devices may be impractical. For instance, a central network device may be connected to a large number of branch network devices (e.g., dozens, hundreds, or more). If such a central network device were to maintain a dedicated BFD session with each branch network device connected to it, the BFD packets sent and received by the central network device could consume significant network resources. Therefore, the use of dedicated BFD sessions can consume network resources that could otherwise be used to send and receive client traffic.

[0057] In some examples, to reduce the consumption of network resources used for performance monitoring, network device 110 may use in-flow performance monitoring. For example, each network device 110 may modify packets carrying client data for a session between client devices 100 to include metadata containing performance metric information. For example, a session between client devices 100A and 100B includes a forward flow originating from client device 100A and destined for client device 100B, and a reverse flow originating from client device 100B and destined for client device 100A. Network device 110A receives a first packet of the forward flow from client device 100A, the first packet including a header and a data payload. Network device 110A modifies the first packet to further include metadata containing first performance information and forwards the modified first packet to network device 110B. Network device 110B can obtain the first performance information from the metadata of the first packet. Furthermore, network device 110B may remove the metadata and forward the first packet to client device 100B (e.g., forward the packet to network device 110C).

[0058] Furthermore, network device 110B receives a second packet from client device 100B in a reverse flow. The second packet includes a header and a data payload. Network device 110B modifies the second packet to further include metadata containing second performance information and forwards the modified second packet to network device 110A. Network device 110A can obtain the second performance information from the metadata of the second packet. Additionally, network device 110A can remove the metadata and forward the second packet to client device 100A.

[0059] In some examples, the metadata includes BFD packets. In some examples, the metadata includes timestamps that network devices 110A and 110B can use to determine performance information. In some examples, the metadata includes measurements of network performance, such as latency, jitter, packet loss, bandwidth, etc. For example, network device 110A modifies the first packet of the forward flow to include metadata specifying a first timestamp indicating the time when network device 110A forwards the first packet to network device 110B. Network device 110B modifies the second packet of the reverse flow to include metadata specifying a second timestamp indicating the time when network device 110B receives the first packet from network device 110A and / or a third timestamp indicating the time when network device 110B forwards the second packet to network device 110A. Network devices 110A and 110B can exchange multiple such modified packets to obtain multiple data points about the performance of link 16 between network devices 110A and 110B. For example, network device 110A can process multiple timestamps to generate metrics for link 16 between network devices 110A and 110B, such as latency, jitter, packet loss, bandwidth, etc. In this way, network devices 110A and 110B can perform performance monitoring of link 16 without interrupting client traffic between client devices 100 or consuming additional network resources by using a dedicated BFD session for performance monitoring.

[0060] Additional information regarding performance monitoring is described in the following: U.S. Patent Application Publication No. US2020 / 0403890 entitled “In-line Performance Monitoring”, published December 24, 2020, and U.S. Patent Application Publication No. 10,200,264 entitled “Link Status Monitoring Based on Packet Loss Detection”, published February 5, 2019, the entire contents of which are incorporated herein by reference.

[0061] As described above, network device 110 forwards each packet of the forward packet stream of session 40 sequentially and along the same forwarding network path. Furthermore, network device 110 forwards each packet of the reverse packet stream of session 40 sequentially and along the same reverse network path (which may be the same as or different from the forward network path). For example, as in... Figure 1 As described in the example, network device 110A forwards network traffic for the forward packet flow of session 40 to network device 110B, and network device 110B then forwards the network traffic for session 40 to network device 110C.

[0062] However, due to dynamic changes in the customer network 140 or the service provider network 150, the path used by network device 110 to forward traffic for session 40 may no longer be suitable. The path may become unsuitable, for example, in the event of a failure in one of the links 16 or an interface of one of the network devices 110, a change in the priority of network traffic for session 40, performance degradation or failure to meet performance requirements of the path traversed by session 40, or changes in the customer network 140 that render the path unsuitable for session 40 (e.g., due to a failure of the client gateway).

[0063] Therefore, in some examples, network device 110 may modify the path of intermediate flow traffic forwarding session 40. For example, one or more network devices 110 may choose to forward packets for session 40 to a different next-hop network device 110 or choose a different ingress or egress interface for sending or receiving packets for session 40. As described above, each network device generates a session identifier for session 40 based on the IP addresses and ports of both the source and destination network devices. Therefore, when modifying the path used by session 40, network device 110 may use dedicated processes to ensure that the stateful nature of session 40 is not lost when migrating session 40 to a new path. In some examples, these dedicated processes may include sharing metadata between network devices on the old path and network devices on the new path. This metadata may include, for example, the source and destination IP addresses and port information of client devices 100A and 100B respectively, and one or more policies, such as network, routing, or security policies, for applying to traffic between client devices 100A and 100B.

[0064] In some examples, network device 110A maintains an action chain for each flow. The action chain includes a chain descriptor specifying the state of the flow. The action chain also includes a series of function blocks, each defining a specific function to be performed as part of routing packets associated with that flow. Network device 110A can operate according to each function block of the action chain to implement routing of packets corresponding to the flow of the action chain. For example, the function blocks of the action chain may specify routing operations including identifiers such as the egress interface of network device 110A, the IP address or port of network device 110A, the next-hop network device 110B, the ingress interface of network device 110B, or the IP address or port of network device 110B.

[0065] Network device 110A can modify the path of session 40 to migrate session 40 in the following ways: Network device 110A deactivates the existing flow for session 40. To deactivate the existing flow for session 40, network device 110A modifies the chain descriptor of the action chain of the flow for session 40 to specify flow invalidation and / or deactivation, so that network device 110A can forward subsequent packets of the flow without using the action chain. Next, network device 110A establishes a new flow for session 40 when the existing flow is deactivated. For example, network device 110A can define a new action chain that reflects the changes in session 40. In some examples, while establishing the new flow, network device 110A can perform dedicated processing for packets received by the flow, such as buffering packets, dropping packets, forwarding packets to the service path, etc. Network device 110A activates the new flow and forwards the packets received by session 40 via the new path of the new flow. For example, to activate the new flow, network device 110A can begin processing packets received by the flow according to the new action chain that reflects the changes in the flow. Network device 110A can subsequently delete old action chains that are now invalid and / or deactivated.

[0066] In some examples, network device 110A may not necessarily need to notify other network devices (e.g., network devices 110B and 110C) of path changes and / or the migration of session 40. As described above, each network device 110 generates a session identifier for session 40 based on the IP addresses and ports of both the source and destination network devices. For example, if network device 110B receives a packet that includes an unknown session identifier (which might be the case for packets from a session undergoing intermediate flow migration), then network device 110B may store the session identifier and perform session-based routing of the packet to treat it as the first packet of the new session.

[0067] Network device 110 can modify the paths used by the forward or reverse packet flow of session 40 in many situations. For example, when network device 110B receives packets for the forward packet flow of session 40 from network device 110A on an incorrect interface of network device 110B, network device 110B can modify the packet flow. In this example, network device 110B can modify the forward packet flow as described above, such that network device 110B receives subsequent packets for the forward packet flow from network device 110A on the correct interface.

[0068] As another example, network device 110 may modify packet flows in response to detecting a routing failure. For example, a routing failure may occur if network device 110A detects a failure of link 16, a failure of an interface of network device 110B, or a failure of network device 110B (e.g., the next-hop network device for the forward packet flow of session 40). In response, for example, network device 110A may modify the packet flow by selecting a different network device 110 (e.g., such as in...). Figure 1 Another hub network device (not shown) can modify the packet flow as the next-hop network device for the forward packet flow of session 40. As another example, network device 110A can modify the packet flow by selecting a different link 16 to network device 110B. Figure 1 (Not shown in the diagram) or the packet flow can be modified by directing the packet flow to different ingress interfaces of network device 110B. In some examples, network device 110 can use BFD to detect path failures, as described above.

[0069] As another example, network device 110 may modify packet flows in response to message collisions. In some examples, network device 110 may modify the forward and reverse flows of a session to define a routing policy based on the latest activity received for the session (e.g., received packets). However, in the case where a first network device and a second network device simultaneously transmit corresponding first and second packets to each other, and the first network device receives the second packet before the second network device receives the first packet, the first network device may establish a flow for the session based on the second packet. Subsequently, the second network device may receive the first packet, which may require the first network device to re-establish the flow for the session. In this example, the first network device may modify the flow as described above according to the routing policy defined by the subsequently received second packet.

[0070] In some examples, network device 110 may modify packet flows in response to detecting additional information related to the network configuration of client device 100. For example, network device 110 may modify packet flows in response to detecting that one or more client devices 100 are using Source Network Address Translation (SNAT).

[0071] Additional information regarding session migration is described in the following: U.S. Patent No. 10,841,206, entitled "Flow Modification Including Shared Context," published November 17, 2020; U.S. Patent Application Publication No. 2021 / 0036953, entitled "Flow Modification Including Shared Context," published February 4, 2021; U.S. Patent Application Publication No. 2021 / 0036953, entitled "Flow Modification Including Shared Context," published February 4, 2021; and U.S. Patent Application Publication No. 2021 / 0036953, entitled "Detecting Source Network Address Transmission in a Communication System," published April 19, 2019. U.S. Patent Application No. 10,257,061 entitled “System”; U.S. Provisional Patent Application No. 63 / 128,672 entitled “Networking Device and Method for Modifying Network Layer Paths in Response to Session State Changes”, filed December 21, 2020; U.S. Patent Application No. 10,432,519 entitled “Packet Redirecting Router”, published October 1, 2019; and U.S. Patent Application No. 10,432,519 entitled “Method and Apparatus for Managing Routing Disruptions in a Computer Network”, published September 24, 2019. The entire contents of U.S. Patent Application No. 10,425,511 for “NETWORK” are incorporated herein by reference.

[0072] According to the technology disclosed herein, network device 110A can perform a “smooth” or gradual failover of network traffic associated with a session (including session 40) and forwarded via a first link to a second link, such that network device 110A iteratively migrates network traffic to increase the number of sessions forwarded from the first link to the second link. For example, network device 110A, as described herein, forwards network traffic of multiple sessions to network device 110B via link 16I. Network device 110A determines that the performance of link 16I does not meet performance requirements. In some examples, the performance requirements are specified by the SLA of the session (or the application or service associated with the session). In some examples, the performance requirements are the performance requirements of the application or service associated with the multiple sessions. Based at least in part on this determination, network device 110A forwards the first portion of the network traffic of the multiple sessions on link 16J instead of link 16I.

[0073] Network device 110A determines that the performance of link 16J meets performance requirements when carrying the first portion of network traffic for multiple sessions. Based on this determination, network device 110A forwards a second, larger portion of network traffic for multiple sessions via link 16J instead of link 16I. Network device 110A continues to iteratively migrate a larger number of sessions from forwarding via link 16I to forwarding via link 16J and tests forwarding via link 16J to ensure compliance with performance requirements, until the entire session's network traffic is forwarded via link 16J. In some examples, network device 110A may increase the proportion of sessions forwarding network traffic via link 16J according to linear, logarithmic, geometric, exponential, or other algorithms. Therefore, using the techniques of this disclosure, when link 16I fails to meet performance requirements, network device 110A may iteratively migrate network traffic from link 16I to link 16J for an increasing number of sessions, so as to progressively test the ability of link 16J to meet performance requirements as the amount of network traffic increases, without overwhelming link 16J with migrated network traffic.

[0074] exist Figure 1 In one example, network device 110A migrates session-specific network traffic from link 16I to link 16J, both of which connect network device 110A to network device 110B. However, in other examples, link 16 can connect to the same or different network devices, such as when both links are connected to network device 110B (e.g., ...). Figure 1 (as shown), or two links connected to different network devices (e.g., links 16I and 16C, connected to network devices 110B and 110C respectively) (in Figure 1 (Not described in the examples).

[0075] Figure 2This is a block diagram illustrating an example network device 110 according to the technology of this disclosure. Typically, network device 110 may be... Figure 1 An example of a network device 110. In this example, network device 110 includes interface cards 226A-226N (“IFC 226”) that receive packets via incoming links 228A-228N (“Incoming Link 228”) and transmit packets via outgoing links 230A-230N (“Outgoing Link 230”). IFC 226 is typically coupled to links 228, 230 via multiple interface ports. Network device 110 also includes a control unit 202 that determines the route of received packets and forwards packets accordingly via IFC 226.

[0076] Control unit 202 may include routing engine 204 and packet forwarding engine 222. Routing engine 204 operates as the control plane of network device 110 and includes an operating system that provides a multitasking environment for executing multiple concurrent processes. Routing engine 204 communicates with other network devices, such as… Figure 1 Network device 110, for establishing and maintaining computer networks, such as Figure 1 The computer network system 2 is used to transmit network traffic between one or more client devices. The routing protocol daemon (RPD) 208 of the routing engine 204 executes software instructions to implement one or more control plane networking protocols 212. For example, protocol 212 may include one or more routing protocols, such as Multiprotocol Label Switching (MPLS) protocol 214, BFD protocol 225, and other routing protocols, such as Internet Group Management Protocol (IGMP) 221 and / or Border Gateway Protocol (BGP) 220, for exchanging routing information with other routing devices and for updating the Routing Information Base (RIB) 206. Protocol 212 may further include one or more communication session protocols, such as TCP, UDP, TLS, or ICMP.

[0077] RIB 206 describes the topology of the computer network in which network device 110 resides, and may also include routes through a shared tree in the computer network. RIB 206 describes the individual routes within the computer network and the appropriate next hop for each route, i.e., along the adjacent routing devices of each route. Routing engine 204 analyzes the information stored in RIB 206 and generates forwarding information stored in Forwarding Information Base (FIB) 224 for forwarding engine 222. FIB 224 may, for example, associate a network destination with a specific next hop and the corresponding IFC 226 and the physical output port for output link 230. FIB 224 may be a radix tree programmed into a dedicated forwarding chip, a series of tables, a complex database, a linked list, a radix tree, a database, a flat file, or various other data structures.

[0078] FIB 224 may also include lookup structures. Given a key value (such as an address), a lookup structure can provide one or more values. In some examples, the one or more values ​​can be one or more next hops. A next hop can be implemented as microcode that performs one or more operations when executed. The one or more next hops can be "linked," such that when executed, a set of linked next hops performs a set of operations for each distinct next hop. Examples of such operations could include applying one or more services to a packet, dropping a packet, and / or forwarding a packet using an interface identified by one or more next hops.

[0079] Session information 235 stores information used to identify sessions. In some examples, session information 235 takes the form of a session table. For example, service information 232 includes one or more entries specifying a session identifier. In some examples, the session identifier includes one or more of the source address, source port, destination address, destination port, or protocol associated with the forward and / or reverse flow of the session. As described above, when routing engine 204 receives data originating from client device 100A and destined for... Figure 1 When a packet in the forward packet flow of client device 100B is received, routing engine 204 determines whether the packet belongs to a new session (e.g., whether it is the "first" or "leading" packet of session 40). To determine whether a packet belongs to a new session, routing engine 204 determines whether session information 235 includes entries corresponding to the source address, source port, destination address, destination port, and protocol of the first packet. If an entry exists, the session is not a new session. If no entry exists, the session is new, and routing engine 204 generates a session identifier for the session and stores the session identifier in session information 235. Routing engine 204 can then use the session identifier stored in session information 235 to identify subsequent packets as belonging to the same session.

[0080] Service information 232 stores information that routing engine 204 can use to identify services associated with a session. In some examples, service information 232 is in the form of a service table. For example, service information 232 includes one or more entries specifying a service identifier and one or more of the source address, source port, destination address, destination port, and protocol associated with the service. In some examples, routing engine 204 can query service information 232 to determine the service associated with a session by querying one or more of the source address, source port, destination address, destination port, or protocol of the session used for the received packet. For example, routing engine 204 can determine the service identifier based on the correspondence between the source address, source port, destination address, destination port, or protocol in service information 232 and the source address, source port, destination address, destination port, or protocol specified by the session identifier. Routing engine 204 retrieves one or more service policies 234 corresponding to the identified service based on the service associated with the packet. Service policies may include, for example, path failover policies, Dynamic Host Configuration Protocol (DHCP) tagging policies, traffic engineering policies, and priority of network traffic associated with the session. The routing engine 204 applies one or more service policies 234 corresponding to the services associated with the group to the group.

[0081] According to the technology disclosed herein, the routing engine 204 of network device 110 includes a smooth failover unit 250 that performs a “smooth” or gradual failover of network traffic associated with a session and forwarded via a first link to a second link, such that network device 110 iteratively migrates network traffic to increase the amount of sessions forwarded from the first link to the second link. Network device 110 can operate as follows: Figure 1 Any network device 110.

[0082] For example, packet forwarding engine 222 forwards network traffic for multiple sessions via outbound link 230A. Smoothing failover unit 250 determines that the performance of outbound link 230A does not meet performance requirements. In some examples, performance requirements are specified by the SLA of a session (or an application or service associated with a session). In some examples, performance requirements are performance requirements of an application or service associated with multiple sessions. In some examples, one or more performance requirements include one or more session-specific requirements for bandwidth of network traffic for each session, latency of network traffic for each session, or jitter of network traffic for each session. In some examples, the performance requirements of a session are SLA requirements for a session (or an application or service corresponding to one or more sessions). In some examples, performance requirements are performance requirements of an application or service associated with multiple sessions. In some examples, smoothing failover unit 250 may use in-flow performance monitoring techniques using BFD as described above to determine the suitability of outbound link 230A. For example, smoothing failover unit 250 may establish a BFD session with another network device 110 via outbound link 230A to determine the performance of outbound link 230A. In some examples, the smooth failover unit 250 can modify the packets of network traffic for one or more sessions to include metadata specifying performance metric information.

[0083] Based at least in part on this determination, the smooth failover unit 250 configures the packet forwarding engine 222 to forward the first portion of network traffic for multiple sessions via outbound link 230B instead of outbound link 230A. The packet forwarding engine 222 forwards the network traffic associated with the first portion of the multiple sessions via outbound link 230B and forwards the remaining portion of network traffic for the multiple sessions via outbound link 230A.

[0084] The smooth failover unit 250 determines that the performance of the outbound link 230B meets the performance requirements when carrying the first portion of network traffic for multiple sessions. Based on this determination, the smooth failover unit 250 configures the packet forwarding engine 222 to forward the second, larger portion of network traffic from the multiple sessions via the outbound link 230B instead of the outbound link 230A. The packet forwarding engine 222 forwards the network traffic associated with the second portion of the multiple sessions via the outbound link 230B and forwards the remaining portion of network traffic from the multiple sessions via the outbound link 230A.

[0085] In some examples, the network traffic in the second part of multiple sessions represents a logarithmic, linear, polynomial, exponential, or factorial increase in the network traffic in the first part of multiple sessions. For example, a network device may use a logarithmic algorithm... Multiple sessions are migrated, where y is the number of sessions to be migrated to the second link, n is an integer representing the iteration count, and x is a scalar constant. As another example, the network device can use a linear algorithm... Multiple sessions are migrated, where y is the number of sessions to be migrated to the second link, n is an integer representing the iteration count, and x is a scalar constant. As another example, the network device can use a polynomial algorithm... Multiple sessions are migrated, where y is the number of sessions to be migrated to the second link, n is an integer representing the iteration count, and x is a scalar constant. As another example, the network device can use an exponential algorithm... Multiple sessions are migrated, where y is the number of sessions to be migrated to the second link, n is an integer representing the iteration count, and x is a scalar constant. As another example, the network device can use a factoring algorithm... Migrate multiple sessions, where y is the number of sessions to be migrated to the second link, n is an integer representing the iteration count, and x is a scalar constant.

[0086] The smooth failover unit 250 continues to iteratively migrate larger numbers of sessions from forwarding via outbound link 230A to forwarding via outbound link 230B, and tests the performance of outbound link 230B to ensure compliance with performance requirements, until the entire session's network traffic is forwarded via outbound link 230B. For example, the smooth failover unit 250 determines that the third performance of outbound link 230B meets the performance requirements when carrying the second portion of network traffic for multiple sessions. Based on this determination, the smooth failover unit 250 configures the packet forwarding engine 222 to forward the third portion of network traffic for multiple sessions, which is larger than the second portion, via outbound link 230B instead of outbound link 230A. The packet forwarding engine 222 forwards the network traffic associated with the third portion of the multiple sessions via outbound link 230B, and forwards the remaining portion of network traffic for the multiple sessions via outbound link 230A. In some examples, the smooth failover unit 250, based at least in part on determining that the outbound link 230B can meet performance requirements after a previous incremental increase in network traffic, according to linear, logarithmic, geometric, exponential or other algorithms (as described above), iteratively increases the proportion of sessions whose network traffic is forwarded via the outbound link 230B for each incremental increase in network traffic.

[0087] In the aforementioned example, the performance requirement is associated with the network performance of the outbound link 230B, which can be determined, for example, by monitoring the bandwidth, latency, or jitter of the outbound link 230B. However, in other examples, the performance requirement is the performance requirement of an application or service associated with multiple sessions. For example, it is possible that the network traffic of multiple sessions is being migrated to an outbound link 230B that meets specific SLA requirements, such as SLA requirements for bandwidth, latency, or jitter. However, the performance of the application or service associated with multiple sessions may be compromised due to transport reasons, such as unnecessary segmentation due to Maximum Transmission Unit (MTU) size issues, Differential Service Code Point (DSCP) rewriting, etc. The technical concept of this disclosure is to evaluate the performance of the outbound link 230B itself, as well as the performance of the application or service associated with multiple sessions, when determining whether the outbound link 230B can meet the performance requirements of a small portion of the network traffic of a session, before migrating network traffic for an increasing number of sessions.

[0088] Therefore, using the techniques disclosed herein, when outbound link 230A fails to meet performance requirements, the smooth failover unit 250 can iteratively migrate network traffic for an increasing number of sessions from outbound link 230A to outbound link 230B, so as the amount of network traffic increases, the ability of outbound link 230B to meet performance requirements is gradually tested without overwhelming outbound link 230B with migrated network traffic.

[0089] In some cases, after a portion of network traffic from multiple sessions has migrated to outbound link 230B, the smooth failover unit 250 may determine that outbound link 230B is not meeting performance requirements. For example, the network traffic from the second portion of the multiple sessions may exceed or exhaust the resources of outbound link 230B, causing a failure or degradation of outbound link 230B's performance. In this case, the smooth failover unit 250 may revert to forwarding only the portion of network traffic from the multiple sessions that was determined to meet performance requirements for outbound link 230B (e.g., reverting to using only the first portion of outbound link 230B from the multiple sessions, relative to the previous example). Once it is determined that outbound link 230B cannot meet the performance requirements of the network traffic from the additional sessions, the smooth failover unit 250 may optionally keep the network traffic from the multiple sessions on the original link (e.g., outbound link 230A), or select a new link to combine (or replace) the use of outbound link 230A and outbound link 230B (e.g., selecting to use outbound link 230C, etc.).

[0090] Therefore, using the techniques disclosed herein, when outbound link 230A fails to meet performance requirements, the smooth failover unit 250 can iteratively migrate network traffic for increasing sessions from outbound link 230A to outbound link 230B, so as the amount of network traffic increases, the ability of outbound link 230B to meet performance requirements is gradually tested without overwhelming outbound link 230B with migrated network traffic.

[0091] In some examples, after migrating network traffic for some sessions, rather than all sessions, from outbound link 230A to outbound link 230B, the smooth failover unit 250 can determine that performance requirements are met for both outbound link 230A and outbound link 230B. For example, migrating the network traffic for the first part of the session may be sufficient to alleviate network congestion on outbound link 230A, allowing its performance to improve sufficiently to meet performance requirements. In this case, the smooth failover unit 250 may proceed in several ways: (1) the smooth failover unit 250 may continue migrating another part (or all) of the network traffic for the session from outbound link 230A to outbound link 230B; (2) the smooth failover unit 250 may maintain the current allocation of session traffic between outbound link 230A and outbound link 230B; or (3) the smooth failover unit 250 may migrate another part (or all) of the network traffic for the session from outbound link 230B back to outbound link 230A. Specific options may be configuration options set by users (such as network administrators).

[0092] For example, after configuring packet forwarding engine 222 to forward the second portion of network traffic for multiple sessions via outbound link 230B instead of outbound link 230A, smooth failover unit 250 determines that the performance of outbound link 230A has recovered to meet performance requirements. Nevertheless, smooth failover unit 250 configures packet forwarding engine 222 to forward the third portion of network traffic for multiple sessions via outbound link 230B instead of outbound link 230A, the third portion being larger than the second portion, even though it determines that the performance of outbound link 230A has recovered to meet performance requirements.

[0093] As another example, after configuring packet forwarding engine 222 to forward the second portion of network traffic for multiple sessions via outbound link 230B instead of outbound link 230A, smooth failover unit 250 determines that the performance of outbound link 230A has recovered to meet performance requirements. In this example, smooth failover unit 250 maintains the distribution of network traffic for multiple sessions between outbound link 230A and outbound link 230B.

[0094] As another example, after configuring packet forwarding engine 222 to forward a second portion of network traffic for multiple sessions via outbound link 230B instead of outbound link 230A, smooth failover unit 250 determines that the performance of outbound link 230A has recovered to meet performance requirements. In this example, smooth failover unit 250 configures packet forwarding engine 222 to return to forwarding another portion (or all) of network traffic for the session via outbound link 230A instead of outbound link 230B.

[0095] Therefore, by increasing the rate of network traffic failover from session to outbound link 230B, while monitoring performance requirements of outbound link 230B each time, the smooth failover unit 250, operating as described herein, ensures that outbound link 230B can meet performance requirements with increased traffic. Thus, the techniques described herein improve the ability of network devices to ensure that links selected for forwarding network traffic meet performance requirements, and improve network throughput by reducing congestion on individual links.

[0096] Figure 3 This is a flowchart illustrating example operations according to the technology disclosed herein. For convenience, relative to... Figure 1 Described Figure 3 .

[0097] like Figure 3 As described in the example, network device 110A forwards network traffic for multiple sessions via a first link 16I (302). Network device 110A determines that the performance of the first link 16I, which forwards the network traffic for the multiple sessions, does not meet performance requirements (304). In some examples, the performance requirements are specified by the SLA of the session (or the application or service associated with the session). In some examples, the performance requirements are the performance requirements of the application or service associated with the multiple sessions. Based at least in part on this determination, network device 110A forwards a first portion of the network traffic for the multiple sessions via a second link 16J instead of the first link 16I (306).

[0098] Network device 110A determines that the performance of a second link 16J, which forwards a first portion of network traffic across multiple sessions, meets performance requirements (308). Based at least in part on this determination, network device 110A forwards a second portion of network traffic across multiple sessions via the second link 16J instead of the first link 16I, wherein the second portion is larger than the first portion (310). In some examples, network device 110A may increase the proportion of sessions whose network traffic is forwarded via link 16I according to linear, logarithmic, geometric, exponential, or other algorithms. Network device 110A may iteratively perform steps 308 and 310, each time increasing the number of sessions whose associated network traffic is forwarded via the second link 16J instead of the first link 16I, until the network traffic of the entire session is forwarded via the second link 16J instead of the first link 16I, or alternatively, until the performance of the second link 16J no longer meets performance requirements, at which point network device 110A may revert to forwarding the previous amount of network traffic for which the second link 16J was determined to meet performance requirements.

[0099] The techniques described in this disclosure can be implemented, at least in part, in hardware, software, firmware, or any combination thereof. For example, aspects of the described techniques can be implemented within one or more processors, including one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or any other equivalent integrated or discrete logic circuitry, and any combination of these components. The terms "processor" or "processing circuitry" generally refer to any of the foregoing logic circuitry, alone or in combination with other logic circuitry, or any other equivalent circuitry. A control unit, including hardware, can also perform one or more of the techniques disclosed herein.

[0100] Such hardware, software, and firmware may be implemented within the same device or in separate devices to support the different operations and functions described in this disclosure. Furthermore, any of the described units, modules, or components may be implemented together or individually as discrete but interoperable logical devices. Describing different features as modules or units is intended to highlight different functional aspects and does not necessarily imply that these modules or units must be implemented by separate hardware or software components. Rather, the functionality associated with one or more modules or units may be performed by separate hardware or software components, or integrated within common or separate hardware or software components.

[0101] The techniques described in this disclosure can also be implemented or encoded in a computer-readable medium (e.g., a computer-readable storage medium) containing instructions. Instructions implemented or encoded in a computer-readable storage medium can cause a programmable processor or other processor to perform the methods, for example, when the instructions are executed. The computer-readable storage medium may include random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, hard disk, CD-ROM, floppy disk, magnetic tape cassette, magnetic media, optical media, or other computer-readable media.

Claims

1. A network device, comprising: A processing circuit that communicates with a storage medium, the processing circuit being configured to: At least in part, based on the determination that the performance of the first link in forwarding network traffic for multiple sessions does not meet the performance requirements, the network traffic for the first portion of the multiple sessions is forwarded via the second link instead of the first link. as well as At least in part based on the performance requirement that the network traffic of the first portion of the plurality of sessions is forwarded via the second link instead of the first link, wherein the second portion is larger than the first portion, the network traffic of the plurality of sessions is forwarded via the second link instead of the first link.

2. The network device according to claim 1, wherein, The processing circuit is further configured to: After forwarding the second portion of network traffic for the plurality of sessions via the second link instead of the first link, and based at least in part on the determination that the performance of the first link has recovered to meet the performance requirements, the distribution of network traffic for the plurality of sessions between the first link and the second link is maintained.

3. The network device according to claim 1, wherein, The processing circuit is further configured to: After forwarding the network traffic of the second portion of the plurality of sessions via the second link instead of the first link, and based at least in part on the determination that the performance of the first link has recovered to meet the performance requirements, forwarding the network traffic of the remaining sessions of the plurality of sessions other than the first and second portions via the first link instead of the second link.

4. The network device according to claim 1, wherein, The network traffic of the second portion of the plurality of sessions represents a logarithmic increase relative to the network traffic of the first portion of the plurality of sessions.

5. The network device according to claim 1, wherein, The network traffic of the second portion of the plurality of sessions represents one of a polynomial increase and an exponential increase relative to the network traffic of the first portion of the plurality of sessions.

6. The network device according to claim 1, wherein, The network traffic of the second portion of the plurality of sessions represents a linear increase relative to the network traffic of the first portion of the plurality of sessions.

7. The network device according to any one of claims 1 to 6, wherein, The processing circuit is further configured to: At least in part based on the performance requirement that the network traffic of the first and second portions of the plurality of sessions is forwarded via the second link instead of the first link, wherein the third portion is larger than the second portion, the network traffic of the plurality of sessions is forwarded via the second link instead of the first link.

8. The network device according to claim 7, wherein, The processing circuitry is configured to forward the third portion of network traffic for the plurality of sessions via the second link instead of the first link, even though it is determined that the performance of the first link has recovered to meet the performance requirements.

9. The network device according to any one of claims 1 to 6, wherein, The processing circuitry is configured to determine the performance of the first link by modifying the network traffic of the plurality of sessions to include metadata containing specified performance metrics.

10. The network apparatus according to any one of claims 1 to 6, wherein, The processing circuitry is configured to determine the performance of the first link by establishing a bidirectional forwarding detection (BFD) session on the first link.

11. A computer networking method, comprising: At least in part, based on the determination that the performance of the first link in forwarding network traffic for multiple sessions does not meet performance requirements, the network device forwards the first portion of the network traffic for the multiple sessions via the second link instead of the first link; as well as The network device forwards a second portion of the network traffic of the plurality of sessions via the second link instead of the first link, at least in part based on the performance requirement that the network traffic of the first portion of the plurality of sessions is forwarded by the second link instead of the first link, wherein the second portion is larger than the first portion.

12. The computer networking method according to claim 11, further comprising: After forwarding the second portion of network traffic for the plurality of sessions via the second link instead of the first link, and based at least in part on the determination that the performance of the first link has recovered to meet performance requirements, the network device maintains the distribution of network traffic for the plurality of sessions between the first link and the second link.

13. The computer networking method according to claim 11, further comprising: After forwarding the network traffic of the second portion of the plurality of sessions via the second link instead of the first link, and based at least in part on the determination that the performance of the first link has recovered to meet the performance requirements, the network device forwards the network traffic of the remaining sessions of the plurality of sessions, excluding the first and second portions, via the first link instead of the second link.

14. The computer networking method according to claim 11, wherein, The network traffic of the second portion of the plurality of sessions represents a logarithmic increase relative to the network traffic of the first portion of the plurality of sessions.

15. The computer networking method according to claim 11, wherein, The network traffic of the second portion of the plurality of sessions represents one of a polynomial increase and an exponential increase relative to the network traffic of the first portion of the plurality of sessions.

16. The computer networking method according to claim 11, wherein, The network traffic of the second portion of the plurality of sessions represents a linear increase relative to the network traffic of the first portion of the plurality of sessions.

17. The computer networking method according to any one of claims 11 to 16, further comprising: At least in part based on the performance requirement that the network device forwards a third portion of the network traffic of the plurality of sessions via the second link instead of the first link, wherein the third portion is larger than the second portion, the network device forwards the network traffic of the plurality of sessions via the second link instead of the first link.

18. The computer networking method according to claim 17, wherein, Even though the network device determines that the performance of the first link has recovered to meet the performance requirements, the network traffic of the third portion of the multiple sessions is forwarded via the second link instead of the first link.

19. The computer networking method according to any one of claims 11 to 16, wherein, Determining the performance of the first link includes modifying the network traffic of the plurality of sessions to include metadata specifying performance metric information.

20. A computer-readable storage medium encoded with instructions for configuring one or more programmable processors to perform a network device according to any one of claims 1 to 10, or to perform a computer networking method according to any one of claims 11 to 19.

Images