Route advertisement using session routing-based path preferences

Abstract

The invention relates to route advertisement using path preferences based on session routing. Some techniques are disclosed for calculating a cost of an advertised route to a next hop network device along a path to a destination based at least in part on a preference of the path. In one example, a computing system computes a cost of routing to a next hop network device along a path to a destination. The calculated cost is based, at least in part, on: (1) a metric of the route; and (2) preconfiguration preference of the path. In some examples, the preference of the path is based, at least in part, on (a) a type of the path compared to other types of other paths to the destination, or (b) a delay of the path compared to other delays of other paths. The computing system sends a route advertisement for the path that includes data indicative of a cost of the route.

Description

[0001] Related Applications

[0002] This application claims the benefit of U.S. patent application No. 18 / 396,482, filed on December 26, 2023, which claims the benefit of U.S. Provisional Application No. 63 / 510,765, filed on June 28, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to computer networking and, more particularly, to routing of network traffic. Background Art

[0004] A computer network is a collection of interconnected computing devices that can exchange data and share resources. Example computing devices include routers, switches, and other layer 2 (L2) network devices that operate in layer 2 (i.e., the data link layer) of the Open Systems Interconnection (OSI) reference model, and layer 3 (L3) network devices that operate in layer 3 (i.e., the network layer) of the OSI reference model. Network devices within a computer network typically include a control unit that provides control plane functions for the network device and forwarding components for routing or switching data units.

[0005] Computing devices may establish a "network session" (also referred to herein as a "session") to enable communication between devices on a computer network. A session may be bidirectional, i.e., a session includes data packets transmitted bidirectionally between a first device and a second device. For example, a session includes a forward data packet flow originating from the first device and destined for the second device and a reverse data packet flow originating from the second device and destined for the first device. The forward and reverse data packet flows of a session are related to each other, i.e., the values ​​specified in the source address and source port of the forward data packet flow are the same as the values ​​specified in the destination address and destination port of the reverse data packet flow, and the values ​​specified in the destination address and destination port of the forward data packet flow are the same as the values ​​specified in the source address and source port of the reverse data packet flow.

[0006] Alternatively, a session may be unidirectional, i.e., the session includes only packets transmitted unidirectionally from the first device to the second device. For example, a session includes a forward packet flow originating from the first device and destined for the second device. A different session may include a reverse packet flow originating from the second device and destined for the first device.

[0007] To establish a session, the computing device may use one or more communication session protocols, including Transmission Control Protocol (TCP), Transport Layer Security (TLS), User Datagram Protocol (UDP), Internet Control Message Protocol (ICMP), and the like. Summary of the invention

[0008] Disclosed herein are techniques for calculating the cost of an advertised route to a next-hop network device along a path to a destination based at least in part on a preference for the path. In one example, a computing system (e.g., a network device) calculates the cost of a route to a next-hop network device along a path to a destination. The calculated cost is based at least in part on: (1) one or more metrics of the route; (2) a preconfigured preference for the path. In some examples, the preference for the path is based at least in part on (a) the type of path compared to other types of other paths to the destination, or (b) the path delay compared to other delays of other paths. In some examples, the preference for the path can be further based on a preference for a particular data center, a particular type of path, or a path prefix. In some examples, the computing system can be configured to use different preferences for network traffic of different applications. The computing system sends a route advertisement via a routing protocol. The route advertisement includes data indicating the cost of the route.

[0009] The technology disclosed herein can provide specific improvements with practical application value for computer-related fields of computer networking. For example, the technology disclosed herein can enable a computing system (e.g., a branch router) to be configured to advertise the cost of the route to the next-hop computing system along each of multiple paths to the destination based on the preference of the corresponding path type. In this way, the branch router can obtain network traffic from the spoke router along a specific desired path. In this way, administrators can configure devices in the network to perform traffic engineering and failover in a more specific and detailed manner, for example, by expressing a preference for a specific type of path, regardless of the specific geographic data center that the path traverses, even when a less desired path through a more preferred data center is available.

[0010] In one example, the present disclosure describes a computer network system comprising a processing circuit and a storage device, wherein the processing circuit is accessible to the storage device and is configured to: calculate a cost of a route based at least in part on: (1) one or more metrics of a route to a next-hop network device along a path to a destination, and (2) a preconfigured preference for the path to the destination; and send a route advertisement of the route via a routing protocol, wherein the route advertisement includes data indicating the cost of the route.

[0011] In another example, the present disclosure describes a method comprising: calculating, by a computing system, a cost of a route based at least in part on: (1) one or more metrics of the route to a next-hop network device along the path to a destination, and (2) a preconfigured preference for the path to the destination; and sending, by the computing system, a route advertisement of the route via a routing protocol, wherein the route advertisement includes data indicating the cost of the route.

[0012] In another example, the present disclosure describes a non-transitory computer-readable medium containing instructions that, when executed, are configured to cause a processing circuit of a computing system to: calculate a cost of a route based at least in part on: (1) one or more metrics of a route to a next-hop network device along the path to a destination, and (2) a preconfigured preference for the path to the destination; and send a route advertisement of the route via a routing protocol, wherein the route advertisement includes data indicating the cost of the route.

[0013] The details of one or more examples of the technology of the present disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the technology will become apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a block diagram illustrating an example computer network system in accordance with various aspects of the disclosed technology;

[0015] Figure 2 is a block diagram illustrating an example computing system in accordance with various aspects of the disclosed technology;

[0016] Figure 3 is a flowchart illustrating example operations according to various aspects of the disclosed technology;

[0017] FIG. 4A to FIG. 4B is a block diagram illustrating an example computer network system in accordance with various aspects of the disclosed technology.

[0018] FIG. 5A to FIG. 5B is a block diagram illustrating an example computer network system in accordance with various aspects of the disclosed technology.

[0019] Figure 6 is a block diagram illustrating an example computer network system in accordance with various aspects of the disclosed technology.

[0020] 7A to 7C is a conceptual diagram illustrating an example definition of path affinity based on each application according to various aspects of the disclosed technology.

[0021] FIG. 8A to FIG. 8B is a block diagram of an example computer network system in accordance with various aspects of the disclosed technology.

[0022] Like reference numerals refer to like elements throughout the drawings and the description. DETAILED DESCRIPTION

[0023] Figure 1 is a block diagram illustrating an example computer network system 2 according to the disclosed technology. Figure 1In the example of , computer network system 2 includes one or more service provider networks, for example, service provider networks 150A-150D (collectively, "service provider networks 150"), which are configured to provide wide area network (WAN) connections to different customer networks 140A-140B (collectively, "customer networks 140"). Network devices 110A-110I (collectively, "network devices 110") of service provider network 150 provide access to service provider network 150 to client devices 100A-100B (collectively, "client devices 100") associated with customer networks 140. In some examples, customer networks 140 are enterprise networks. For ease of illustration, customer network 140A is described as having a single client device 100A and customer network 140B is described as having a single client device 100B, but each customer network 140 may include any number of client devices.

[0024] In some examples, customer network 140 may be an L2 computer network, where the number after the layer refers to the corresponding layer in the Open Systems Interconnection (OSI) model. L2 is also referred to as the "data link layer" in the OSI model, and the term L2 may be used interchangeably with the "data link layer" in the present disclosure. Typically, customer network 140 includes a number of client devices 100, each of which may communicate with each other through a service provider network 150, as described below. Communication links 16A-16H (collectively referred to as links "16") may be Ethernet, asynchronous transfer mode (ATM), or any other suitable network connection. In other examples, customer network 140 may be an L3 network. Common L3 operations include operations performed according to L3 protocols (e.g., Internet Protocol (IP)). In the OSI model, L3 is also referred to as the "network layer", and the term L3 may be used interchangeably with the "network layer" in the present disclosure.

[0025] The network devices 110 may be implemented using any network device, such as a switch, a router, a gateway, or other suitable network device that can send and receive network traffic. For example, the customer networks 140 may be networks of geographically separated sites of an enterprise. Each customer network 140 may include additional customer devices, such as one or more non-edge switches, routers, hubs, gateways, security devices such as firewalls, intrusion detection and / or intrusion prevention devices, servers, computer terminals, laptops, printers, databases, wireless mobile devices such as cellular phones or personal digital assistants, wireless access points, bridges, cable modems, application accelerators, or Figure 1 Other routers not shown. Figure 1 The configuration of the computer network system 2 shown is only an example. For example, the computer network system 2 may include any number of customer networks 140. However, for ease of description, Figure 1 Only customer networks 140A-140B are shown.

[0026] Service provider network 150 represents one or more publicly accessible computer networks owned and operated by one or more service providers. Figure 1 140, but in other examples, the computer network system 2 may instead include a single service provider network that provides connectivity between customer networks 140. The service provider is typically a large telecommunications entity or company. Each service provider network 150 is typically a large L3 computer network. Each service provider network 150 is an L3 network in the sense that it natively supports L3 operations as described in the OSI model.

[0027] Although not shown, each service provider network 150 may be connected to one or more networks managed by other providers, and may therefore form part of a large-scale public network infrastructure, such as the Internet. Thus, customer networks 140 may be viewed as edge networks of the Internet. Each service provider network 150 may provide computing devices (e.g., client devices 100 and 102) within customer network 140 with access to the Internet, and may allow computing devices within customer network 140 to communicate with each other.

[0028] Although additional network devices are not shown for ease of explanation, system 2 may include additional network and / or computing devices, such as one or more additional switches, routers, hubs, gateways, security devices such as firewalls, intrusion detection and / or intrusion prevention devices, servers, computer terminals, laptops, printers, databases, wireless mobile devices such as cellular phones or personal digital assistants, wireless access points, bridges, cable modems, application accelerators or other routers. In addition, although the elements of system 2 are shown as being directly connected, one or more additional network elements may be included along any network link 16 so that the network elements of system 2 communicate but are not directly connected.

[0029] Each service provider network 150 typically provides a number of residential and business services to customer networks 140, including residential and business-type data services (which are often referred to as "Internet services" because they allow access to a collection of publicly accessible networks known as the Internet), residential and business-type telephone and / or voice services, and residential and business-type television services.

[0030] In some examples, the network device 110 can implement a stateful, session-based routing scheme that enables each network device 110 to independently perform path selection and traffic engineering. The use of session-based routing can enable the network device 110 to avoid using a centralized controller, such as a software-defined network (SDN) controller, to perform path selection and traffic engineering. In this way, the network device 110 may be more efficient and scalable for large networks where the use of an SDN controller is not feasible. In addition, the use of session-based routing can enable the network device 110 to avoid using tunnels, thereby saving a large amount of network resources by eliminating the need to perform encapsulation and decapsulation at the tunnel endpoints. In some examples, the network device 110 implements session-based routing as a secure vector routing (SVR) provided by Juniper Networks. In some examples, the network device 110 includes an Ethernet over SVR (EoSVR) router.

[0031] exist Figure 1 In the example of , client device 100A of system 2 establishes session 40 (indicated by a dashed line) with client device 100B. Network device 110 facilitates establishment of session 40 by transmitting network traffic between client device 100A and client device 100B. In some examples, client device 100A may be considered a "source" device because client device 100A initiates session 40 between client device 100A and client device 100B, e.g., client device 100A is the "source" of packets of a forward flow of the session. Session 40 includes a forward packet flow originating from client device 100A and destined for client device 100B and a reverse packet flow originating from client device 100B and destined for client device 100A. The forward flow of session 40 traverses a first path, e.g., including client device 100A, network devices 110A-110I, and client device 100B.

[0032] In some examples, the network device 110 may extend the session 40 as an L3 session on the service provider network 150 according to one or more L3 communication session protocols (including the transmission control protocol (TCP) or the user datagram protocol (UDP), etc.). In order to establish the session 40 according to TCP so that data can be exchanged according to TCP, the network device 100A and the network device 100B perform a three-way handshake. The network device 100A sends a first data packet including a "SYN" flag to the network device 100B. The network device 100B confirms the receipt of the first data packet by responding to the network device 100A with a second data packet including a "SYN-ACK" flag. The network device 100A confirms the receipt of the second data packet by responding to the network device 100B with a third data packet including an "ACK" flag. After sending the third data packet, the session 40 is established according to TCP, and the network devices 100A and 100B can exchange data with each other via the session 40. For additional example information about TCP, see "TRANSMISSION CONTROL PROTOCOL," Request for Comments (RFC) 793, Internet Engineering Task Force (IETF), September 1981, available at https: / / tools.ietf.org / html / rfc793, the entire contents of which are incorporated herein by reference.

[0033] UDP is a connectionless protocol, i.e., network device 100A does not verify whether network device 100B is able to receive data before sending data. To establish session 40 according to UDP, network device 100A sends a first data packet to network device 100B. When network device 100A receives any data packet from network device 100B, session 40 can be considered to be "established" according to UDP, which means that network device 100B successfully received the first data packet from network device 100A, responded, and network device 100A was able to receive a response from network device 100B. For additional example information about UDP, see "User Datagram Protocol", RFC 768, IETF, August 28, 1980, available at https: / / tools.ietf.org / html / rfc768, the entire contents of which are incorporated herein by reference.

[0034] exist Figure 1In some examples, when network device 110A receives a packet of a forward packet flow originating from client device 100A and destined for client device 100B, network device 110A determines whether the packet belongs to a new session (e.g., is the “first” packet or the “leader” packet of session 40). In some examples, network device 110A determines whether session information (e.g., source address, source port, destination address, destination port, and protocol) in the first packet matches an entry in the session table.

[0035] If no such entry exists, network device 110A determines that the data packet belongs to a new session and creates an entry in the session table. In addition, if the data packet belongs to a new session, network device 110A may generate a session identifier for session 40. The session identifier may include, for example, the source address and source port of client device 100A, the destination address and destination port of client device 100B, and the protocol used by the first data packet. Network device 110A may use the session identifier to identify subsequent data packets belonging to the same session.

[0036] In some examples, network device 110 performs stateful routing for session 40. For example, network device 110 may forward each packet of a forward packet flow of session 40 in sequence and along the same forward network path. As described herein, the "same" forward path may mean the same network device 110 that forms a segment or at least a portion of a path between a device that originates a packet and a device to which the packet is destined (and not necessarily the entire network path between the device that originates the packet and the device to which the packet is destined). In addition, network device 110 forwards each packet of a return flow of session 40 in sequence and along the same return network path. The forward network path of the forward packet flow of session 40 and the return network path of the return packet flow of session 40 may be the same path, or different paths. By ensuring that each packet of the flow is forwarded in sequence and along the same path, network device 110 maintains the state of the entire flow at each network device 110, thereby enabling the use of stateful packet services, such as deep packet inspection (DPI).

[0037] exist Figure 1In an example of , a stateful routing session may be established from network device 110A (which may be an ingress router) through intermediate network devices (e.g., network devices 110B-110H) to network device 110I (which may be an egress router). In this example, network device 110A determines that the first data packet is an unmodified data packet and is the first data packet of a new session 40. Network device 110A modifies the first data packet to include metadata specifying a session identifier. In some examples, the metadata may be inserted after the header of the modified first data packet. For example, the session identifier may include an original source address, a source port, a destination address, a destination port, a protocol, a service, and / or a tenant. Network device 110A replaces the header of the modified first data packet to specify a source address (i.e., the address of network device 110A), a source port (i.e., the port through which network device 110A forwards the modified first data packet toward client device 100B), a destination address (i.e., the address of the next hop to which network device 110A forwards the first data packet (e.g., the address of network device 110B)), and a destination port (i.e., the port of the next hop to which network device 110A forwards the first data packet (e.g., the port of network device 110B)).

[0038] The network device 110A may further identify the network service associated with the session 40. For example, the network device 110A may compare one or more of the source address, source port, destination address, or destination port of the session with the service address and port information table to identify the service associated with the session. Examples of network services include Hypertext Transfer Protocol (HTTP), firewall services, proxy services, packet monitoring or measurement services, etc. For example, the network device 110A may determine that the forward packet flow of the session 40 specifies the destination address and destination port assigned to the client device 100B. Thereafter, the network device 110A may store the association between the session 40 and the identified network service. As another example, if the source port and / or destination port of the session 40 is 80, the network device 110A may determine that the session 40 is associated with the HTTP service. In other examples, the network device 110A may determine that one or more of the source address, source port, destination address, and destination port of the session 40 belongs to an address or port block indicating that a specific service is associated with the session 40.

[0039] In some examples, network device 110A uses the network service determined for session 40 to select a forwarding path for forwarding the first packet and each subsequent packet of the forward packet flow of session 40 to client device 100B. In this way, network device 110A can perform service-specific path selection to select a network path that best suits the service requirements. In contrast to network topologies that use an SDN controller to perform path selection, each network device 110 performs path selection. In addition, the use of session-based routing enables each network device 110 to make routing decisions at the service or application level, in contrast to traditional routers that can only make routing decisions at the flow level.

[0040] In addition, network device 110A may store the session identifier of session 40 so that upon receiving subsequent packets of session 40, network device 110A may identify the subsequent packets as belonging to the same session 40 and forward the subsequent packets along the same path as the first packet. In some examples, network device 110A, upon receiving these subsequent packets, may forward the subsequent packets without metadata specifying the session identifier.

[0041] Intermediate network device 110B receives the modified first data packet from network device 110A. Network device 110B determines whether the modified first data packet includes metadata specifying a session identifier. In response to determining that the modified first data packet includes metadata specifying the session identifier, intermediate network device 110B determines that network device 110B is not an ingress device, such that network device 110B does not attach the metadata specifying the session identifier.

[0042] As described above with respect to network device 110A, network device 110B determines whether a packet belongs to a new session (e.g., is the "first" packet or the "leader" packet of a session) by determining whether the source address, source port, destination address, destination port, and protocol of the first packet match an entry in the session table. If no such entry exists, network device 110B may determine that the packet belongs to a new session and create an entry in the session table. Additionally, if the packet belongs to a new session, network device 110B may generate a session identifier for the session. The session identifier used by network device 110B to identify the session of the first packet may be different from the session identifier used by network device 110A to identify the same session of the first packet because each network device 110A, 110B uses the header source address, source port, destination address, and destination port of the first packet to generate the session identifier, and this header information may be modified by each preceding network device 110 as each network device 110 forwards the first packet along the forwarding path. Additionally, each network device 110 may store the header information to identify a previous network device 110 (or “waypoint”) and a next network device 110 (or “waypoint”) so that each network device 110 may reconstruct the same forward path and reverse path for each subsequent packet of the session.

[0043] Network device 110B may replace the header of the modified first packet to specify a source address that is the address of network device 110B, a source port that is the port via which network device 110B forwards the modified first packet to client device 100B, a destination address that is the address of the next hop to which network device 110B forwards the first packet (e.g., the address of network device 110C for session 40 along the first path), and a destination port that is the port of the next hop to which network device 110B forwards the first packet (e.g., the port of network device 110C). Network device 110B forwards the modified first packet to network device 110C. In addition, network device 110B may store a session identifier for the session so that upon receiving subsequent packets of the session, network device 110B may identify the subsequent packets as belonging to the same session and forward the subsequent packets along the same path as the first packet.

[0044] Subsequent intermediate network devices 110C-110H may process the modified first packet in a similar manner as network devices 110A and 110B, such that network device 110 forwards subsequent packets of the session along the same path as the first packet. In addition, each network device 110 may store a session identifier for the session, which may include an identification of a previous network device 110 along the network path. Thus, each network device 110 may forward packets of a reverse packet flow of the session back to client device 100A along the same network path using the session identifier.

[0045] A network device 110 that can forward packets of a forward packet flow of a session to a destination of the packet flow can be referred to as an egress or "end point" router. In the foregoing example, network device 110I can be referred to as an end point device because network device 110I can forward packets to client device 100B. Network device 110I receives a modified first packet that includes metadata (e.g., original source address, source port, destination address, and destination port) that specifies a session identifier. Network device 110I determines the modified first packet as being destined for a service that terminates at network device 110I by determining that the destination source address and destination source port specified in the metadata of the modified leading packet correspond to a destination reachable by network device 110I (e.g., client device 100B). Network device 110I restores the original first packet by removing the metadata from the modified first packet and using the metadata to modify the header of the first packet to specify the original source address and source port of client device 100A and the destination address and destination port of client device 100B. Network device 110I forwards the restored first packet to client device 100B. Thus, use of session-based routing may form a series of waypoints (eg, network devices 110) interconnected by path "segments" (eg, end-to-end routing vectors between each waypoint).

[0046] For additional information on session-based routing and SVR, see U.S. Patent No. 9,729,439, entitled “COMPUTERNETWORK PACKET FLOW CONTROLLER,” issued on August 8, 2017; “Secure Vector Routing (SVR),” draft-menon-svr-00, Internet-Draft, Internet Engineering Task Force (IETF), October 1, 2021, available at https: / / datatracker.ietf.org / doc / draft-menon-svr / 00 / ; “Secure Vector Routing (SVR),” draft-menon-svr-01, Internet-Draft, Internet Engineering Task Force (IETF), March 29, 2022, available at https: / / datatracker.ietf.org / doc / draft-menon-svr / 01 / ; and “Secure Vector Routing (SVR)”, draft-menon-svr-02, Internet-Draft, Internet Engineering Task Force (IETF), September 20, 2022, available at https: / / datatracker.ietf.org / doc / draft-menon-svr / 02 / ; the entire contents of each of which are incorporated herein by reference.

[0047] Bidirectional Forwarding Detection (BFD) is a network protocol for detecting failures in a bidirectional path between two network devices (e.g., link 16B between network devices 110A and 110B). BFD provides low-overhead, short-time detection of failures in a link between two network devices. In addition, BFD provides a single mechanism that can be used for liveness detection in any media, at any protocol layer, with a wide range of detection times and overheads, avoiding the emergence of different methods between adjacent devices. BFD operates on top of any data protocol (network layer, link layer, tunnel, etc.) forwarded between two network devices. Typically, BFD operates in unicast, point-to-point mode. BFD packets are carried as payloads of any encapsulation protocol suitable for the medium and network.

[0048] According to BFD, network devices 110A and 110B establish a session via link 16B. Typically, network devices 110A and 110B establish and terminate a BFD session via a three-way handshake. Typically, network devices 110A and 110B can declare link 16B operational only after bidirectional communication is established between network devices 110A and 110B. However, this does not exclude the use of unidirectional links. For example, link 16B may represent a first unidirectional link from network device 110A to network device 110B and a second unidirectional link from network device 110B to network device 110A.

[0049] Once the BFD session is established, network devices 110A and 110B periodically transmit BFD packets over link 16B. Each network device 110A, 110B estimates the speed at which BFD packets are sent and received in order to negotiate the speed of fault detection with the peer network devices 110A, 110B. In some examples, network devices 110A and 110B may modify these estimates in real time to adapt to network congestion, delay or bandwidth changes, or other anomalies. This allows the use of a shared medium between fast network devices and slow network devices, while allowing fast network devices to detect faults more quickly and allowing slow network devices to participate in fault detection.

[0050] BFD can operate in two modes: asynchronous mode and demand mode. In asynchronous mode, if one of network devices 110A and 110B stops receiving BFD packets for a certain period of time (the length of time is determined by negotiation as described above), network devices 110A and 110B can assume that link 16B (or the components, devices or paths that constitute link 16B) has failed. In demand mode, network devices 110A and 110B can negotiate not to send periodic BFD packets to reduce overhead. This assumes that network devices 110A and 110B have another way to verify connectivity to each other, such as through the physical layer. However, if necessary, either network device 110A, 110B can still send BFD packets.

[0051] In addition, either network device 110A or 110B can use an echo function. When the function is activated, network device 110A sends an echo packet stream to network device 110B. Network device 110B transmits the echo packet back to network device 110A via the forwarding plane of network device 110B in response. Network device 110A can use the echo function to test the forwarding path to network device 110B, and vice versa. For descriptions of other example information about BFD, see “Bidirectional Forwarding Detection (BFD)”, RFC5880, IETF, June 2010, available at https: / / datatracker.ietf.org / doc / html / rfc5880; and “Bidirectional Forwarding Detection (BFD) for IPv4 and IPv6 (Single Hop)”, RFC5881, IETF, June 2010, available at https: / / datatracker.ietf.org / doc / rfc5881 / , the entire contents of which are incorporated herein by reference.

[0052] Network device 110 creates a separate BFD session for each communication path and data protocol used between the two network devices. For example, to perform fault detection along the entire path of session 40 between network devices 110A and 110I, a different BFD session may be established along each link 16, e.g., a first BFD session is established along the first link between network devices 110A and 110B, a second BFD session is established along link 16C between network devices 110B and 110C, and so on.

[0053] In some examples, it may not be feasible to use a dedicated BFD session between two network devices. For example, a hub router may be connected to many spoke routers (e.g., dozens, hundreds, or more routers). If such a hub router were to maintain a dedicated BFD session with each spoke router to which the hub router is connected, the BFD packets sent and received by the hub router may consume a large amount of network resources. Therefore, using a dedicated BFD session may consume network resources that could otherwise be used to send and receive customer traffic.

[0054] In some examples, in order to reduce network resource consumption for performance monitoring, network device 110 may use in-flow performance monitoring. For example, each network device 110 may modify a data packet carrying client data of a session between client devices 100 to include metadata including performance information. For example, a session between client device 100A and client device 100B includes a forward flow originating from client device 100A and destined for client device 100B and a reverse flow originating from client device 100B and destined for client device 100A. Network device 110A receives a first data packet of the forward flow from client device 100A, the first data packet including a header and a data payload. Network device 110A modifies the first data packet to further include metadata including first performance information, and forwards the modified first data packet to network device 110B. Network device 110B may obtain the first performance information from the metadata of the first data packet. In addition, network device 110B may remove the metadata and forward the first data packet toward client device 100B (e.g., by forwarding the data packet to network device 110C).

[0055] In addition, network device 110B receives a second data packet of a reverse flow from client device 100B, the second data packet including a header and a data payload. Network device 110B modifies the second data packet to further include metadata including second performance information, and forwards the modified second data packet to network device 110A. Network device 110A may obtain the second performance information from the metadata of the second data packet. In addition, network device 110A may remove the metadata and forward the second data packet toward client device 100A.

[0056] In some examples, the metadata includes a BFD packet. In some examples, the metadata includes a timestamp that network devices 110A, 110B can use to determine performance information. In some examples, the metadata includes measurements of network performance, such as measurements of delay, jitter, packet loss, bandwidth, etc. For example, network device 110A modifies a first packet of a forward flow to include metadata specifying a first timestamp indicating the time when network device 110A forwarded the first packet to network device 110B. Network device 110B modifies a second packet of a reverse flow to include metadata specifying a second timestamp and / or a third timestamp, the second timestamp indicating the time when network device 110B received the first packet from network device 110A, and the third timestamp indicating the time when network device 110B forwarded the second packet to network device 110A. Network devices 110A and 110B may exchange multiple such modified packets to obtain multiple data points about the performance of link 16B between network devices 110A and 110B. For example, network device 110A may process multiple timestamps to generate metrics of link 16B between network devices 110A and 110B, such as delay, jitter, packet loss, bandwidth, etc. In this manner, network devices 110A and 110B may perform performance monitoring on link 16B without disrupting customer traffic between customer devices 100 or consuming additional network resources by using a dedicated BFD session for performance monitoring.

[0057] For more information on performance monitoring, see U.S. patent application publication No. 2020 / 0403890, entitled “IN-LINE PERFORMANCE MONITORING,” published on December 24, 2020; U.S. patent application publication No. 10,200,2640, entitled “LINK STATUS MONITORING BASED ON PACKET LOSS DETECTION,” published on February 5, 2019; and U.S. patent application publication No. 17 / 449,311, entitled “OPPORTUNISTIC MESH FOR SOFTWARE-DEFINED WIDE AREA NETWORK (SD-WAN),” filed on September 29, 2021, the entire contents of each of which are incorporated herein by reference.

[0058] According to the technology of the present disclosure, a network device (e.g., Figure 1 ) in the network devices 110 in the example embodiment calculates a cost of an advertised route to a next-hop network device along a path to one or more destinations based at least in part on a preference for the path.

[0059] For example, about Figure 1As previously described, session 40 includes forward packet flow 50A originating from client device 100A and destined for client device 100B and reverse packet flow 50B originating from client device 100B and destined for client device 100A. Network device 110B may have two (or more) paths along which packets of forward packet flow 50A are forwarded. A first path to client device 100B exists from network device 110B, traversing link 16C, to network device 110C, traversing link 16E, to network device 110E, and then to network devices 110F-110I. A second path to client device 100B is from network device 110B, traversing link 16H, to network device 110D, traversing link 16D, to network device 110E, and then to network devices 110F-110I. In some examples, the first path may be a multi-protocol label switching (MPLS) path and the second path may be a broadband path.

[0060] Network device 110C calculates a cost for a first route along a first path to a destination (client device 100B) to a next-hop network device (e.g., network device 110E). The calculated cost is based at least in part on: (1) a metric for the route; and (2) a preconfigured preference for the path. The metric may include one or more of delay, jitter, and packet loss for link 16E to network device 110E.

[0061] In some examples, the preference for the path is based at least in part on the type of path compared to other types of paths to the client device 100B. For example, an MPLS type path may be more preferred and a broadband type path may be less preferred. In some examples, the preference for the first path is based at least in part on the latency of the first path to the client device 100B compared to the latency of other paths (e.g., the second path) to the client device 100B. The network device 110C sends a route advertisement of the first route to the network device 110B via a routing protocol. The route advertisement includes data indicating the cost of the route. In some examples, the routing protocol is BGP.

[0062] The network device 110D can calculate the cost of a second route to a next-hop network device (e.g., network device 110E) along the second path to the destination (client device 100B) in a similar manner. The network device 110D sends a route advertisement of the second route to the network device 110B through the routing protocol. Since the first path is better than the second path, the advertised cost of the first path is less than the advertised cost of the second path.

[0063] After receiving the packets of the forward packet flow 50A, the network device 110B identifies the client device 100B as the destination of the packets of the forward packet flow 50A. The network device 110B selects the route with the lowest cost to forward the packets of the forward packet flow 50A using the route advertisement of the first route from the network device 110C and the route advertisement of the second route from the network device 110D. Therefore, since the preference for a particular path is considered when calculating the cost of the first route and the second route, the network device 110B decides to forward the packets of the forward packet flow 50A using the first path, and forwards the packets of the forward packet flow 50A to the network device 110C.

[0064] Figure 2 is a block diagram illustrating an example computing system 200 according to the techniques of the present disclosure. In general, computing system 200 may be Figure 1 The computing system 200 includes a processing circuit 202 for executing any one or more of an application 222, a routing component 250, or any other computing device described herein. Other examples of the computing system 200 may be used in other situations to implement the techniques of this disclosure.

[0065] Although for the purposes of this example Figure 2 200, but a computing device operating according to the techniques of the present disclosure may be any component or system including one or more processors or other suitable computing environment for executing software instructions, and need not include, for example, Figure 2 206; and in some examples, components such as storage device 208 may not be co-located or in the same chassis as other components. In some examples, computing system 200 may be implemented as a virtualized network function (VNF). In some examples, one or more aspects of computing system 200 may be operated as one or more containers, or as one or more applications within a virtual machine of a network function virtualization (NFV) platform using, for example, virtual input and output (VirtIO) and single root input / output virtualization (SR-IOV) network virtualization technologies, or on a bare metal server. In some examples, computing system 200 is a physical network device, such as a switch, a router, a gateway, or other device that sends and receives network traffic. In some examples, computing system 200 may be a distributed system.

[0066] like Figure 2As shown in the example of , the computing system 200 includes a processing circuit 202, one or more input devices 204, one or more communication units 206, one or more output devices 212, one or more storage devices 208, and one or more user interface (UI) devices 210. In one example, the computing system 200 also includes one or more applications 222 and an operating system 216 that can be executed by the computing system 200. Each of the components 202, 204, 206, 208, 210, and 212 is connected (physically, communicatively, and / or operatively) for inter-component communication. In some examples, the communication channel 214 may include a system bus, a network connection, an inter-process communication data structure, or any other method for transmitting data. As an example, the components 202, 204, 206, 208, 210, and 212 can be connected via one or more communication channels 214.

[0067] In one example, processing circuitry 202 is configured to implement functions and / or process instructions for execution within computing system 200. In some examples, processing circuitry 202 includes one or more hardware-based processors. For example, processing circuitry 202 may access storage device 208 and may be capable of processing instructions stored in storage device 208. Examples of processing circuitry 202 may include any one or more of a microprocessor, a controller, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or equivalent discrete or integrated logic circuits.

[0068] One or more storage devices 208 may be configured to store information within the computing system 200 during operation. In some examples, the storage device 208 is described as a computer-readable storage medium. In some examples, the storage device 208 includes temporary storage, which means that the primary purpose of the storage device 208 is not long-term storage. In some examples, the storage device 208 includes volatile memory, which means that the storage device 208 does not maintain the stored contents when the computer is turned off. Examples of volatile memory include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and other forms of volatile memory. In some examples, the storage device 208 stores program instructions executed by the processing circuit 202. In one example, the storage device 208 is used by software or applications operating on the computing system 200 to temporarily store information during program execution.

[0069] In some examples, storage device 208 also includes one or more computer-readable storage media. Storage device 208 can be configured to store a larger amount of information than volatile memory. Storage device 208 can also be configured for long-term storage of information. In some examples, storage device 208 includes a non-volatile storage element. Examples of such non-volatile storage elements include magnetic hard disks, optical disks, floppy disks, flash memory, or various forms of electrically programmable memory (EPROM) or electrically erasable programmable (EEPROM) memory.

[0070] In some examples, computing system 200 also includes one or more communication units 206. In one example, computing system 200 uses communication unit 206 to communicate with external devices via one or more networks, such as one or more wired / wireless / mobile networks. Communication unit 206 may include a network interface, such as an Ethernet card, an optical transceiver, a radio frequency transceiver, or any other type of device capable of sending and receiving information. Other examples of such network interfaces may include 3G / 4G / 5G and WiFi radios. In some examples, communication unit 206 may include multiple high-speed network interface cards. In some examples, computing system 200 uses communication unit 206 to communicate with external devices. For example, computing system 200 uses communication unit 206 to communicate with external devices via a network to which communication unit 206 is connected. Figure 1 Link 16 and Figure 1 100 and / or the client device 100.

[0071] In one example, computing system 200 also includes one or more user interface devices 210. In some examples, user interface device 210 is configured to receive input from a user through tactile, audio, or video feedback. Examples of user interface device 210 include a presence-sensitive display, a mouse, a keyboard, a voice response system, a camera, a microphone, or any other type of device for detecting commands from a user. In some examples, the presence-sensitive display includes a touch-sensitive screen. In some examples, a user such as an administrator of service provider network 150 can enter configuration data for computing system 200.

[0072] One or more output devices 212 may also be included in the computing system 200. In some examples, the output device 212 is configured to provide output to a user using tactile, audio, or visual stimulation. In one example, the output device 212 includes a presence-sensitive display, a sound card, a video graphics adapter card, or any other type of device for converting signals into an appropriate form understandable to a human or machine. Additional examples of the output device 212 include a speaker, a cathode ray tube (CRT) monitor, a liquid crystal display (LCD), or any other type of device capable of producing understandable output to a user.

[0073] Computing system 200 may include operating system 216. In some examples, operating system 216 controls the operation of components of computing system 200. For instance, in one example, operating system 216 facilitates communication of one or more applications 222 with processing circuitry 202, communication unit 206, storage device 208, input device 204, user interface device 210, and output device 212. Applications 222 may also include program instructions and / or data that may be executed by computing system 200.

[0074] In some examples, processing circuit 202 executes routing component 250, which determines the route of a received data packet and forwards the data packet accordingly. Routing component 250 is coupled to other routers (e.g., Figure 1 110) to establish and maintain a computer network (e.g., Figure 1 A computer network system 2) for transmitting network traffic between one or more client devices. The routing protocol daemon (RPD) 254 of the routing component 250 can execute software instructions to implement one or more control plane network protocols 256. For example, the protocol 256 can include one or more routing protocols, such as the Internet Group Management Protocol (IGMP) 221 and / or the Border Gateway Protocol (BGP) 220, for exchanging routing information with other routing devices and updating the routing information base (RIB) 252, the Multi-Protocol Label Switching (MPLS) protocol 215, and other routing protocols. The protocol 256 can also include one or more communication session protocols 223, such as TCP, UDP, TLS, or ICMP. The protocol 256 further includes BFD224.

[0075] Routing information 252 may describe the topology of the computer network in which computing system 200 resides, and may also include routes through shared trees in the computer network. Routing information 252 may describe various routes within the computer network and the appropriate next hop for each route, i.e., the adjacent routing devices along each route. Routing information 252 may be programmed into a dedicated forwarding chip, a series of tables, a complex database, a linked list, a radix tree, a database, a flat file, or a variety of other data structures.

[0076] Session information 235 stores information for identifying a session. In some examples, session information 235 is in the form of a session table. For example, service information 232 includes one or more entries that specify a session identifier. In some examples, the session identifier includes one or more of a source address, a source port, a destination address, a destination port, and a protocol associated with a forward packet flow and / or a reverse packet flow of the session. As described above, when routing component 250 receives a packet from a session, Figure 1When a packet of the forward packet flow from client device 100A to client device 100B is received, routing component 250 determines whether the packet belongs to a new session (e.g., is the "first" packet or the "leader" packet of session 40). To determine whether the packet belongs to a new session, routing component 250 determines whether session information 235 includes an entry corresponding to the source address, source port, destination address, destination port, and protocol of the first packet. If an entry exists, the session is not a new session. If no entry exists, the session is new, and routing component 250 generates a session identifier for the session and stores the session identifier in session information 235. Thereafter, routing component 250 can use the session identifier stored in session information 235 to identify subsequent packets belonging to the same session.

[0077] The service information 232 stores information that the routing component 250 can use to identify the service associated with the session. In some examples, the service information 232 is in the form of a service table. For example, the service information 232 includes one or more entries specifying a service identifier and one or more of a source address, source port, destination address, destination port, and protocol associated with the service. In some examples, the routing component 250 can query the service information 232 with one or more of the source address, source port, destination address, destination port, and session protocol of the received data packet to determine the service associated with the session. For example, the routing component 250 can determine the service identifier based on the correspondence between the source address, source port, destination address, destination port, or protocol in the service information 232 and the source address, source port, destination address, destination port, or protocol specified by the session identifier. The routing component 250 retrieves one or more service policies 234 corresponding to the identified service based on the service associated with the data packet. The service policy can include, for example, a path failover policy, a dynamic host configuration protocol (DHCP) tag policy, a traffic engineering policy, a priority of network traffic associated with the session, and the like. Routing component 250 applies one or more service policies 234 corresponding to the services associated with the data packet to the data packet.

[0078] In some examples, computing system 200 can be used as Figure 1 Any network device 110 in the operation. Figure 2 In the following example, the computing system 200 operates as the network device 110A. The computing system 200 receives a data packet from the client device 100A through the communication unit 206, and the destination of the data packet is the client device 100B. In response to receiving the data packet, the computing system 200 generates an L3 data packet, which includes an L3 header and metadata, wherein the metadata includes a source IP address and a source port of the client device 100A and a destination IP address and a destination port of the client device 100B.

[0079] In some examples, the metadata includes a session identifier. A session identifier is a unique identifier for a session that includes a first packet flow originating from a first client device (e.g., client device 100A) and destined for a second client device (e.g., client device 100B) and a second packet flow originating from the second client device and destined for the first client device. Typically, a session identifier includes a 5-tuple, such as a source IP address and source port of the first client device, a destination IP address and destination port of the second client device, and a network protocol used by the session.

[0080] In this example, the data packet is the first data packet of a plurality of data packets of a session. In response to receiving the first data packet, computing system 200 may generate a session identifier for the session between client device 100A and client device 100B, and store the session identifier in session information 235 (e.g., to store metadata of the L3 data packet). As described above, computing system 200 may generate an L3 data packet including an L3 header and metadata based on the first data packet. Computing system 200 forwards the L3 data packet to the next hop network device, namely, network device 110B, via communication unit 206.

[0081] For subsequent packets, computing system 200 may determine that the subsequent packets belong to the same session as the first packet based on information (e.g., 5-tuple) of the subsequent packets. For example, in response to receiving a second packet of a session, computing system 200 may perform a lookup of session information 235 and determine that the second packet belongs to a session having a corresponding entry in session information 235 based on a source address, source port, destination address, destination port, and / or protocol specified by the second packet.

[0082] In response to determining that the subsequent data packet belongs to the same session as the first data packet, computing system 200 can generate a subsequent L3 data packet for the subsequent data packet that includes an L3 header but does not include metadata (because the session identifier associated with the given session has been stored, for example, by each subsequent network device that received the first data packet).

[0083] The computing system 200 may forward the subsequent L3 packets toward the same next-hop network device 110B. The intermediate network devices (e.g., network devices 110B-110H) may receive the subsequent L3 packets, identify the session associated with the subsequent L3 packets from the L3 headers of the subsequent L3 packets, and perform session-based routing on the subsequent L3 packets using the stored metadata. In this manner, the computing system 200 may modify only the first L3 packet of the multiple L3 packets generated for the session to include the metadata, thereby avoiding including the metadata in each of the multiple L3 packets generated for the session, while still ensuring that each of the multiple L3 packets is forwarded along the same path (e.g., forwarded to the same next-hop network device 110), and the egress network device (e.g., network device 110I) may recover the original packet from the L3 packets.

[0084] refer to Figure 2 In the following example, the computing system 200 operates as the network device 110I and receives the L3 packet modified as described above from the network device 110H. For example, the computing system 200 receives the L3 packet including the L3 header and metadata through the communication unit 206.

[0085] If the L3 packet is the first L3 packet of multiple L3 packets for a session, computing system 200 may generate a session identifier for the session between client device 100A and client device 100B using the metadata and store the session identifier in session information 235 (e.g., to store the metadata of the L3 packet). In response to determining that the destination of the packet is a client device connected to network device 110I, computing system 200 may restore the original packet using the metadata of the L3 packet. For example, computing system 200 may modify the header of the packet using the metadata to specify the original source address, source port, destination address, and destination port. Computing system 200 may then forward the restored packet to client device 100B.

[0086] The computing system 200 may further receive subsequent L3 packets that do not include metadata in the multiple L3 packets. For these subsequent L3 packets, the computing system 200 may determine that the subsequent L3 packets belong to the same session as the first L3 packet based on the L3 header (e.g., the source IP address and source port of the previous hop network device and the destination IP address and destination port of the network device 110I). The computing system 200 may determine the original destination port and destination address of the packet from the metadata of the stored first L3 packet. The computing system 200 may forward the subsequent packets to the client device 100A via the communication unit 206. In this way, the computing system 200 may only receive the first L3 packet that specifies metadata, while subsequent L3 packets do not include such information. Therefore, the network device 110 described herein may avoid including metadata in each of the multiple L3 packets generated for the session, while still ensuring that each of the multiple L3 packets is forwarded along the same path (e.g., forwarded to the same next hop network device 110).

[0087] According to the technology of the present disclosure, the path preference 260 of the routing component 250 stores preferences for different paths. In some examples, the path preference 260 stores a path identifier, a path type, and a path type preference for each of the multiple paths. In some examples, the path type can be MPLS, broadband, long-term evolution (LTE), 5G wireless, or other types of network paths. In some examples, the path preference 260 stores the latency of the path for each of the multiple paths.

[0088] The routing component 250 calculates the cost of the route to the next hop network device along the path for each path to the destination. The calculated path is based at least in part on one or more metrics of the path to the next hop network device along the path. In addition, the calculated path is based at least in part on preconfigured preferences for the path to the destination. For example, the routing component 250 obtains one or more metrics of the path to the next hop network device. The routing component 250 determines the cost of the route based at least in part on the one or more metrics of the route to the next hop network device. The routing component 250 adjusts the cost of the route based at least in part on the preconfigured preferences for the path to the destination.

[0089] In some examples, the preconfigured preferences for the paths include a preconfigured preference for the path type of the paths compared to the path types of other paths to the destination. In some examples, the preconfigured preferences for the paths are based on the delay of the paths to the destination compared to the delay of other paths to the destination. In some examples, the preferences are a ranking of paths to the same destination based on the delay of each path.

[0090] The routing component 250 only sends routing advertisements with the routing protocol. The routing advertisements include data indicating the cost of the route.

[0091] In some examples, routing component 250 calculates the cost of the route of the first path as described above, and sends a route advertisement in response to detecting a failure of the second path to the destination. In this way, the network device receiving the route advertisement can failover the failed second path, thereby failing over the network traffic to the most preferred path maintained after the failure of the second path. Typically, at least one network device forming the first path is different from at least one network device forming the second path.

[0092] In some examples, routing component 250 calculates a cost for each of a plurality of routes, each route being routed to a corresponding next-hop network device along a corresponding path in a plurality of different paths to a destination. The plurality of paths traverse a plurality of geographically independent data centers. The cost for each of the plurality of routes is based at least in part on: (1) one or more metrics of the route to the corresponding next-hop network device along the corresponding path in the plurality of paths to the destination, and (2) a preconfigured preference for the corresponding path in the plurality of paths to the destination.

[0093] In some examples, routing component 250 determines a first cost for a first route based at least in part on one or more first metrics of a first route to a first next-hop network device along a first path to the destination. Routing component 250 determines a second cost for a second route based at least in part on one or more second metrics of a second route to a second next-hop network device along a second path to the destination. In this example, the first cost of the first route is greater than the second cost of the second route.

[0094] The routing component 250 adjusts a first cost of a first route based at least in part on a preconfigured preference of the first path to the destination, and adjusts a second cost of a second route based at least in part on a preconfigured preference of the second path to the destination. In this example, the preconfigured preference of the first path indicates that the first path is more preferred and the preconfigured preference of the second path indicates that the second path is less preferred. In addition, the adjusted first cost of the first route is less than the adjusted second cost of the second route. Therefore, a network device that receives the first route advertisement and the second route advertisement may choose to forward network traffic along the first route because the first route is more preferred than the second route, even though the performance of the first route may be worse than the performance of the second route as shown by metrics of the first route and the second route considered separately.

[0095] Figure 3 is a flowchart showing an example of operation according to the disclosed technology. Figure 3 refer to Figure 1 Give a description.

[0096] For example, about Figure 1 As previously described, session 40 includes a forward packet flow 50A originating from client device 100A and destined for client device 100B and a reverse packet flow 50B originating from client device 100B and destined for client device 100A. Network device 110B may have two (or more) paths to forward packets of forward packet flow 50A. A first path for client device 100B is from network device 110B, across link 16C, to network device 110C, across link 16E, to network device 110E, and then to network devices 110F-110I. A second path for client device 100B is from network device 110B, across link 16H, to network device 110D, across link 16D, to network device 110E, and then to network devices 110F-110I. In some examples, the first path may be a multi-protocol label switching (MPLS) path and the second path may be a broadband path.

[0097] like Figure 3 As shown in the example of FIG. 1 , network device 110C calculates a cost (300) of a first path to a destination (client device 100B) to a next-hop network device (e.g., network device 110E) along the first path. The calculated cost is based at least in part on: (1) a metric of the route; and (2) a preconfigured preference for the path. The metric may include one or more of delay, jitter, or packet loss of link 16E to network device 110E.

[0098] In some examples, the preference of a path is based at least in part on the type of path compared to other types of paths to the client device 100B. For example, an MPLS type path may be more preferred, while a broadband type path may be less preferred. In some examples, the preference of a first path is based at least in part on the latency of the first path to the client device 100B compared to the latency of other paths (e.g., the second path) to the client device 100B.

[0099] Network device 110C sends a route advertisement of a first route to network device 110B via a routing protocol (302). The route advertisement includes data indicating a cost of the route. In some examples, the routing protocol is BGP.

[0100] FIG. 4A to FIG. 4B 4 is a block diagram showing an example of a computer network system 400 according to the disclosed technology. More specifically, Figure 4A It is shown that according to the technology of this disclosure, the network device 110 expresses the preference for the path according to the transmission affinity. Figure 4B It is shown that according to the techniques of this disclosure, network device 110 expresses preferences for paths based on transport affinity on a per-application basis.

[0101] like Figure 4A As shown in the example of , the computer network system 400 includes a branch network device 410A, a peer network device 410B in a data center 402A, and a peer network device 410C in a data center 402B. The branch network device 410A is connected to the peer network device 410B in the data center 402A via an MPLS path 416A and a broadband path 416C. The branch network device 410A is connected to the peer network device 410C in the data center 402B via an MPLS path 416B and a broadband path 416D (paths 416A-416D are collectively referred to as "paths 416"). In some examples, the branch network device 410A can make multi-host connections with multiple peer network devices in the data center 402A, the data center 402B, or the data centers 402A and 402B.

[0102] Typically, when routing network traffic to a destination, a customer may prefer data center 402B over data center 402A. Peer network devices 410B, 410C and branch network device 410A may advertise available routes through corresponding data centers 402A, 402B via BGP, and indicate corresponding MPLS paths 416A, 416B as primary paths and corresponding broadband paths 416C, 416D as secondary paths. Thus, during normal operation, branch network device 410A selects MPLS path 416B to forward packets of a forward packet flow to peer network device 410C of data center 402B, while peer network device 410C selects MPLS path 416B to forward packets of a reverse packet flow to branch network device 410A.

[0103] In some cases, MPLS path 416B to data center 402B may fail. Branch network device 410A can (1) failover the forward packet flow to broadband path 416D to data center 402B as a secondary path; or (2) use MPLS path 416A provided by data center 402A.

[0104] Regarding the above solution (1), if the broadband path 416D is still operating, the BGP communication can continue. Therefore, the branch network device 410A and the peer network device 410C can failover the forward packet flow and the reverse data flow from the MPLS path 416B (e.g., the primary path) of the data center 402B to the broadband path 416D (e.g., the secondary path) of the data center 402B. However, using the broadband path 416D may be slower or unreliable than using the MPLS path 416A provided by the data center 402A.

[0105] Alternatively, with respect to option (2) above, branch network device 410A may choose to use MPLS path 416A to peer network device 410B provided by data center 402A for packets of the forward packet flow. However, since data center 402B is configured as preferred over data center 402A, and the secondary route (e.g., broadband path 416D) of data center 402B is still active, peer network device 410B may forward the packets to peer network device 410C to continue to use broadband path 416D as a secondary route to the failed primary route (MPLS path 416B) to the preferred data center 402B, rather than migrating to use MPLS path 416A through data center 402A. A mismatch between branch network device 410A using MPLS path 416A through data center 402A and peer network device 410C using broadband path 416D through data center 402B may create a loop, which may increase latency, causing branch network device 410A to be unable to recognize the reverse packet flow received from peer network device 410C, causing network address translation (NAT) errors or other problems.

[0106] Based on the best path, the branch network device 410A needs to operate the advertisement of 172.20.1.0 / 24 so that the data packets of the data center enter the path preferred according to the path preference. Figure 4A , although the route 0.0.0.0 / 0 is advertised from both data centers 402A, 402B, the peer network devices 410B, 410C want to prefer the data center with the best path preference. Traditionally, there is no way to express this intent. This is a higher level intent, not just on a per application basis. Once a data center is selected, the application's traffic will follow the application's corresponding path preference.

[0107] like Figure 4B As shown in the example, according to the technology of the present disclosure, the branch network device 410A that can perform session-based routing can consider the preference of a specific path type when calculating the cost of the route along the path to the next-hop network device 410B, 410C for route announcement via a routing protocol (e.g., BGP). Figure 4B In the example of FIG. 4 , the branch network device 410A may apply a preference for a specific path type based on each application program, thereby allocating a corresponding path type preference to network traffic of each application type in a plurality of different application types.

[0108] For example, a branch network device 410A may be connected to a destination via multiple data centers 402A, 402B. Each data center 402A, 402B may include multiple peer network devices 410B, 410C, wherein each peer network device 410B, 410C provides a separate path 416A, 416B, 416C, and 416D to the destination to the branch network device 410A, each path traversing multiple different network devices. In addition, each path may be of a different type, such as MPLS, LTE, or broadband. All paths across different data centers can be organized into path groups based on path type, regardless of the specific data center each path traverses.

[0109] In operation, a network device (e.g., branch network device 410A) determines one or more available paths 416 to a destination and a preference for each determined path 416. Paths 416 may include, for example, paths that traverse different peer routers of data centers 402A, 402B or paths that traverse different data centers 402A, 402B. For each determined path 416, branch network device 410A calculates one or more metrics (e.g., packet loss, delay, jitter) of the route along path 416 to the next-hop network device 410B, 410C, and calculates the cost of the route based on the one or more metrics. Branch network device 410A adjusts the cost of the route based on the preference for the path type of the corresponding path 416 of the route. Branch network device 410A advertises the route for the next-hop network device 410B, 410C along the path via a routing protocol (e.g., BGP), and the route includes information specifying the adjusted cost of the route.

[0110] Thus, each application can follow the above overall strategy to determine data center 402 and then select the best path to that data center using the calculated cost. In addition, different applications may have different preferences. For example, an application may define a path prefix of 200.0.1.0 / 24 and specify, in order from most preferred to least preferred, MPLS path 416B to data center 402B, MPLS path 416A to data center 402A, broadband path 416D to data center 402B, and broadband path 416C to data center 402A. In short, each application may have different path preferences. High-level path preferences determine which data center 402 to select. Alternatively, a driver can be used to select a path preference for a given application. Once the application's network traffic data center is determined, the application path preference is respected. More details on the driver application will be provided in the following sections. Fig. 7A discussed in.

[0111] In this manner, the techniques of the present disclosure may enable a network device to prioritize bidirectional network traffic failover over a path of a more preferred type before attempting to use a path of a second, less preferred path type. For example, a customer may wish to use an MPLS path before using a broadband path, regardless of which data center the MPLS path traverses. The techniques of the present disclosure may enable a customer to transfer forward and reverse packet flows from a first MPLS path through a first data center to a second MPLS path through a second data center before failing over to a broadband path through the first data center.

[0112] FIG. 5A to FIG. 5B is a block diagram of an example computer network system 500 according to the techniques of the present disclosure. More specifically, Figure 5A Techniques in accordance with this disclosure are shown in which network device 110 expresses preferences for paths based on datacenter affinity. Figure 5B Techniques in accordance with the present disclosure are shown in which network device 110 expresses preferences for paths based on datacenter affinity on a per-application basis.

[0113] like Figure 5A As shown in the example of FIG. 5 , the computer network system 500 includes a branch network device 510A, a peer network device 510B in a data center 502A, and a peer network device 510C in the data center 502B. The branch network device 510A is connected to the peer network device 510B in the data center 502A via an MPLS path 516A and a broadband path 516C. The branch network device 510A is connected to the peer network device 510C in the data center 502B via an MPLS path 516B and a broadband path 516D (paths 516A-516D are collectively referred to as "paths 516").

[0114] Based on datacenter affinity, the branch network device 510 can manipulate the advertisement of prefix 172.20.1.0 / 24 so that datacenter packets arrive at the hub as the preferred datacenter. Even if address 0.0.0.0 / 0 is advertised from both datacenters 502A, 502B, the branch network device 510 will still prefer the primary datacenter. Traditionally, there is no way to express this intent. This is a higher level intent, not just on a per-application basis. Once a datacenter is selected, the application traffic follows the corresponding application path preference.

[0115] like Figure 5BAs shown in the example of , according to the technology of the present disclosure, the branch network device 510A that can perform session-based routing is configured to consider the preference for a specific data center when calculating the cost of the route along the path to the next-hop network device 510B, 510C for routing announcement via a routing protocol (e.g., BGP). In addition, Figure 5B In the example of FIG. 5 , the branch network device 510A can apply a preference for a specific path type based on each application, thereby allocating a corresponding data center preference to network traffic of each application type in a plurality of different application types. Figure 5B In the example of , the branch network device 510A is configured according to the techniques of the present disclosure and may be a computing system, for example, Figure 2 The computing system 200 in FIG.

[0116] Therefore, in Figure 5B In the example of FIG. 5 , the branch network device 510A can consider the preference for a specific data center for each application to select the preferred data center 502, and then use the calculated cost to select the best path to that data center. In addition, different applications may have different preferences. The branch network device 510A can use the higher-level path preferences to determine the primary data center for the data centers 502A, 502B. Once the branch network device 510A determines the primary data center, the application path preferences are respected.

[0117] Figure 6 600 is a block diagram of a computer network system according to a technical example of the present disclosure. More specifically, Figure 6 An example is shown in which the network device 110 expresses a preference for a path based on the data center (DC) affinity of each prefix. In some examples, the branch network device 610A can apply a preference for a specific prefix based on each application, thereby allocating a corresponding prefix preference to network traffic of each application type in a plurality of different application types.

[0118] like Figure 6 As shown in the example of FIG. 6 , the computer network system 600 includes a branch network device 610A, a peer network device 610B in a data center 602A, and a peer network device 610C in a data center 602B. The branch network device 610A is connected to the peer network device 610B in the data center 602A via an MPLS path 616A and a broadband path 616C. The branch network device 610A is connected to the peer network device 610C in the data center 602B via an MPLS path 616B and a broadband path 616D (paths 616A-616D are collectively referred to as "paths 616").

[0119] Based on the data center affinity for each prefix, branch network device 610A wants to prefer data center 602A for prefix 200.1.0.0 / 24 and data center 602B for prefix 30.30.1.0 / 24. Traditionally, there is no way to express this intent. This is a higher level intent, not just on a per-application basis. Once a data center is selected for a prefix, applications within that prefix follow that path preference.

[0120] like Figure 6 As shown in the example, according to the technology of the present disclosure, the branch network device 610A that can perform session-based routing can consider the preference for specific path types when calculating the cost of the route along the path to the next-hop network device 610B, 610C for route announcement via a routing protocol (e.g., BGP).

[0121] 7A to 7C is a conceptual diagram illustrating various example definitions of path affinity based on each application according to various aspects of the disclosed technology. For convenience, 7A to 7C It is for FIG. 4A to FIG. 4B However, the technology of the present disclosure can also be applied to Figure 1 Computer network system 2. FIG. 5A to FIG. 5B Computer network system 500 or Figure 6 Computer network system 600.

[0122] Fig. 7A An example is described in which three different applications (Application 1, Application 2, and Application 3) each specify a preference for a different path on a per-prefix basis. Fig. 7A It is shown that three applications can be associated with different path preferences based on each prefix. In some examples, Fig. 7A A common method is described for expressing an intent to select a data center based on a per-prefix path preference to determine a data center. A network device (e.g., branch network device 410A) can consider preferences for specific path types when calculating the cost of a route along a path to a next-hop network device 610B, 610C for route advertisement via a routing protocol. In addition, the advertised routes can be specific to specific applications, so that when branch network device 410A advertises the cost of the route for traffic for the corresponding application, different preferences for different paths by application 1, application 2, and application 3 can be expressed. The following describes a per-prefix path preference method for 3 different types of applications. These definitions can be defined like applications, but can be identified as driver applications.

[0123] Application 1 (Parent 1)

[0124] Prefix: 0.0.0.0 / 0

[0125] Application Strategy - SLA

[0126] Path preference (sorting)

[0127] a)Mpls->DC1

[0128] b)Mpls->DC2

[0129] c)Broadband->DC1

[0130] d)Broadband->DC2

[0131] Application 2 (Parent 2)

[0132] Prefix: 200.1.0.0 / 24

[0133] Application Strategy - SLA

[0134] Path preference (sorting)

[0135] a)Mpls->DC2

[0136] b)Mpls->DC1

[0137] c)Broadband->DC2

[0138] d)Broadband->DC1

[0139] Application 3 (Parent 3)

[0140] Prefix: 80.0.1.0 / 24

[0141] Application Strategy - SLA

[0142] Path preference (sorting)

[0143] a)Broadband->DC1

[0144] b)Broadband->DC2

[0145] FIG. 7B to FIG. 7C Two different application groups are shown, namely application group 1 and application group 2. FIG. 7B to FIG. 7C As shown, Fig. 7A Applications in the application can be grouped into a set of applications (e.g., "application groups"), where each application group is assigned a different path preference on a per-prefix basis. In addition, a parent application can inherit the per-prefix path preference of each application group, and each child application can inherit the per-prefix path preference of each application group, or different path preferences can be configured on a per-prefix basis for each application group.

[0146] Figure 7B An example is described in which the parent application and the child application of application group 1 can specify different preferences for different paths based on each prefix. Figure 7B As shown in the example, child applications can inherit the path preferences of the parent application or specify their own specific path preferences. The following describes the path preference method based on each prefix for the parent application A, which has 4 child applications A1, A2, A3 and A4.

[0147] Application A (Parent)

[0148] Prefix: 0.0.0.0 / 0

[0149] Application Strategy - SLA

[0150] Path preference (sorting)

[0151] a)Mpls->DC1

[0152] b)Mpls->DC2

[0153] c)Broadband->DC1

[0154] d)Broadband->DC2

[0155] Application A1 (Sub1)

[0156] Prefix: 80.0.1.0 / 24

[0157] Application Strategy - SLA

[0158] Path preference (sorting)

[0159] a)Broadband->DC1

[0160] b)Broadband->DC2

[0161] Application A2 (Sub-2) Prefix: 0.0.0.0 / 0, Port 443 Application Policy - SLA

[0162] Path preference (sorting)

[0163] a)Broadband->DC1

[0164] b)Broadband->DC2

[0165] c)LTE->DC1

[0166] d) LTE->DC2 Application A3 (child)

[0167] Prefix: 0.0.0.0 / 0

[0168] UDP Port 53

[0169] Access wifi

[0170] Application Strategy - SLA

[0171] Path preference (sorting)

[0172] c)Broadband->DC1

[0173] d)Broadband->DC2

[0174] Application A4 (child)

[0175] Prefix: 0.0.0.0 / 0

[0176] Port 443,22

[0177] Access lan tenant

[0178] Application Strategy - SLA

[0179] Path preference (sorting)

[0180] a)Mpls->DC1

[0181] b)Mpls->DC2

[0182] Figure 7C Another example is described in which the parent application and child applications of application group 2 can specify preferences for different paths based on each prefix. Figure 7C The path preference configuration of the application in application group 2 can be done with Figure 7B The path preference configurations of the applications in application group A are similar or different. Figure 7C As shown in the example, child applications can inherit the path preferences of the parent application or specify their own specific path preferences. The following is a per-prefix path preference method for a parent application B, with 2 child applications B1 and B2.

[0183] Application B (Parent)

[0184] Prefix: 200.1.0.0 / 24

[0185] Application Strategy - SLA

[0186] Path preference (sorting)

[0187] a)Mpls->DC2

[0188] b)Mpls->DC1

[0189] c)Broadband->DC2

[0190] d)Broadband->DC1

[0191] Application B1 (Sub1)

[0192] Prefix: 200.0.1.3 / 32

[0193] Application Strategy - SLA

[0194] Path preference (sorting)

[0195] a)Broadband->DC1

[0196] b)Broadband->DC2

[0197] Application B2 (Sub 2)

[0198] Prefix: 200.0.1.0 / 24, port 22, policy-SLA

[0199] Path preference (sorting)

[0200] a)Mpls->DC1

[0201] b)Mpls->DC2

[0202] FIG. 8A to FIG. 8B 8 is a block diagram showing different use cases of an example of a computer network system 800 according to the disclosed technology. More specifically, FIG8 shows an example including multiple branch network devices 810A and 810D.

[0203] like FIG. 8A to FIG. 8B As shown, the computer network system 800 includes a branch network device 810A, a peer network device 810B in a data center 802A, a peer network device 810C in a data center 602B, and a branch network device 810D. The branch network device 810A is connected to the peer network device 810B in the data center 802A via a broadband path 816C, and is connected to the peer network device 810C in the data center 802B via a broadband path 816D. The branch network device 810D is connected to the peer network device 810B in the data center 802A via MPLS 816A, and is connected to the peer network device 810C in the data center 802B via an MPLS path 816B (paths 816A-816D are collectively referred to as "paths 816")

[0204] Fig. 8A A first use case of summarizing routes to a Hub LAN BGP network device is described. In this example, it is desired to provide an intent that only the summarized routes need to be advertised to the Hub LAN. Individual routes from spokes (e.g., branch network devices 810A, 810D) will be advertised to the Hub LAN. However, this does not define the advertisement of individual routes from the hub to the spokes (which are not summarized). Branch network devices 810A (e.g., Fig. 8AThe prefix of the spoke 1 in the network is advertised to the branch network device 810D (for example, Fig. 8A 2 in the spoke). Branch network device 810D (e.g., Fig. 8A The prefix of the spoke 2 in the hub is advertised to the branch network device 810A (for example, Fig. 8A The radial path in 1).

[0205] Figure 8B A second use case is described for aggregating routes from spoke network devices to spoke network devices. In this example, it is desired to provide an intent that the aggregated routes only need to be advertised to the spoke network devices (e.g., branch network device 810A and branch network device 810D). The individual routes from the spokes will be advertised to the Hub LAN (and not aggregated). However, this does not define the individual routes that are advertised to the Hub LAN (these routes are not aggregated). Branch network device 810A (e.g., Figure 8B The prefix of the spoke 1) in the hub is used as a summary prefix to the branch network device 810D (for example, Figure 8B The branch network device 810D (for example, Figure 8B The prefix of the spoke 2 in the hub is used as a summary prefix to the branch network device 810A (for example, Figure 8B The spoke 1) in the network is notified.

[0206] In the third use case, the administrator can turn off spoke-to-spoke advertising. It is desirable to provide an intent that routes learned from a spoke network device are not advertised to other spoke network devices. In this case, the spoke network device receives routes only from the Hub LAN. Spokes will not learn any routes from other spokes. This is to prevent spoke-to-spoke traffic. This implementation is typically done to meet certain corporate compliance requirements.

[0207] The techniques described in this disclosure may be implemented, at least in part, in hardware, software, firmware, or any combination thereof. For example, various aspects of the described techniques may be implemented in one or more processors, including one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or any other equivalent integrated or discrete logic circuits and any combination of these components. The term "processor" or "processing circuit" may generally refer to any of the aforementioned logic circuits (alone or in combination with other logic circuits) or any other equivalent circuit. A control unit including hardware may also perform one or more techniques of the present disclosure.

[0208] Such hardware, software, and firmware may be implemented within the same device or within separate devices to support the various operations and functions described in this disclosure. In addition, any described units, modules, or components may be implemented together or separately as discrete but interoperable logic devices. Describing different features as modules or units is intended to highlight different functional aspects and does not necessarily mean that these modules or units must be implemented by separate hardware or software components. On the contrary, the functions associated with one or more modules or units may be performed by separate hardware or software components, or integrated in common or separate hardware or software components.

[0209] The techniques described in the present disclosure may also be embodied or encoded in a computer-readable medium containing instructions, such as a computer-readable storage medium. The instructions embedded or encoded in the computer-readable storage medium may cause a programmable processor or other processor to perform the method, for example, when the instructions are executed. The computer-readable storage medium may include a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a hard disk, a CD-ROM, a floppy disk, a cassette, a magnetic medium, an optical medium, or other computer-readable medium.

Claims

1. A computer network system, comprising a processing circuit and a storage device, wherein: The processing circuit is capable of accessing the storage device and is configured to: calculating a cost of a route based at least in part on: (1) one or more metrics of a route to a next-hop network device along a path to a destination, and (2) a preconfigured preference for the path to the destination; as well as A route advertisement for the route is sent via a routing protocol, wherein the route advertisement includes data indicating a cost for the route.

2. The computer network system according to claim 1, wherein: To calculate the cost of the route, the processing circuit is configured to: determining a cost of the route to the next hop network device based at least in part on the one or more metrics of the route; as well as A cost of the route is adjusted based at least in part on the preconfigured preference for the path to the destination.

3. The computer network system according to claim 1, wherein: The preconfigured preference for the path includes a preconfigured preference for a path type for the path compared to path types of other paths to the destination.

4. The computer network system according to claim 1, wherein: The preconfigured preference of the path is based on a delay of the path to the destination compared to path types of other paths to the destination.

5. The computer network system according to claim 1, in, The paths include a first path to the destination, and The processing circuit is configured to perform the calculating and the sending in response to detecting a failure of a second path to the destination, wherein at least one network device forming the first path is different from at least one network device forming the second path.

6. The computer network system according to any one of claims 1 to 5, in, To calculate the cost of the route, the processing circuit is configured to calculate a cost for each of a plurality of routes, each route following a corresponding path of a plurality of different paths to the destination to a corresponding next-hop network device, wherein at least two of the plurality of paths pass through different data centers of a plurality of geographically separated data centers, and wherein the cost of each of the plurality of routes is based at least in part on (1) one or more metrics of the route to the corresponding next-hop network device along the corresponding path of the plurality of paths to the destination, and (2) a preconfigured preference of the corresponding path of the plurality of paths to the destination.

7. The computer network system according to any one of claims 1 to 5, in, To calculate the cost of the route, the processing circuit is configured to: determining a first cost of a first route based at least in part on one or more first metrics of a first route to a first next-hop network device along a first path to the destination; determining a second cost for a second route based at least in part on one or more second metrics for a second route to a second next-hop network device along a second path to the destination, wherein the first cost for the first route is greater than the second cost for the second route; adjusting the first cost of the first route based at least in part on a preconfigured preference of the first path to the destination; and The second cost of the second route is adjusted based at least in part on a preconfigured preference of the second path to the destination, wherein the preconfigured preference of the first path indicates that the first path is more preferred, the preconfigured preference of the second path indicates that the second path is less preferred, and wherein the adjusted first cost of the first route is less than the adjusted second cost of the second route.

8. The computer network system according to claim 7, in, The first path comprises a Multi-Protocol Label Switching (MPLS) path, and wherein the second path comprises a broadband path.

9. The computer network system according to any one of claims 1 to 5, wherein: The routing protocol includes the Border Gateway Protocol BGP.

10. A computer networking method, comprising: calculating, by a computing system, a cost of a route based at least in part on: (1) one or more metrics of a route to a next-hop network device along a path to a destination, and (2) a preconfigured preference for the path to the destination; as well as A route advertisement for the route is sent by the computing system via a routing protocol, wherein the route advertisement includes data indicating a cost of the route.

11. The computer networking method according to claim 10, wherein: Calculating the cost of the route includes: determining a cost of the route to the next hop network device based at least in part on the one or more metrics of the route; and A cost of the route is adjusted based at least in part on the preconfigured preference for the path to the destination.

12. The computer networking method according to claim 10, wherein: The preconfigured preference for the path includes a preconfigured preference for a path type for the path compared to path types of other paths to the destination.

13. The computer networking method according to claim 10, wherein: The preconfigured preference of the path is based on a delay of the path to the destination compared to path types of other paths to the destination.

14. The computer networking method according to claim 10, in, The paths include a first path to the destination, and Wherein the calculating and the sending are in response to detecting a failure of a second path to the destination, wherein at least one network device forming the first path is different from at least one network device forming the second path.

15. The computer networking method according to any one of claims 10 to 14, in, Calculating the cost of the route includes calculating a cost for each of a plurality of routes, each route along a corresponding path of a plurality of different paths to the destination to a corresponding next-hop network device, wherein at least two of the plurality of paths pass through different data centers of a plurality of geographically separated data centers, and wherein the cost of each of the plurality of routes is based at least in part on (1) one or more metrics of the route to the corresponding next-hop network device along the corresponding path of the plurality of paths to the destination, and (2) a preconfigured preference of the corresponding path of the plurality of paths to the destination.

16. The computer networking method according to any one of claims 10 to 14, wherein: Calculating the cost of the route includes: determining a first cost of a first route based at least in part on one or more first metrics of a first route to a first next-hop network device along a first path to the destination; determining a second cost for a second route based at least in part on one or more second metrics for a second route to a second next-hop network device along a second path to the destination, wherein the first cost for the first route is greater than the second cost for the second route; adjusting the first cost of the first route based at least in part on a preconfigured preference of the first path to the destination; and The second cost of the second route is adjusted based at least in part on a preconfigured preference of the second path to the destination, wherein the preconfigured preference of the first path indicates that the first path is more preferred, the preconfigured preference of the second path indicates that the second path is less preferred, and wherein the adjusted first cost of the first route is less than the adjusted second cost of the second route.

17. The computer networking method according to claim 16, in, The first path comprises a Multi-Protocol Label Switching (MPLS) path, and wherein the second path comprises a broadband path.

18. The computer networking method according to any one of claims 10 to 14, wherein: The routing protocol includes the Border Gateway Protocol BGP.

19. A computer-readable storage medium encoded with instructions for causing one or more programmable processors to be configured as a computer network system according to any one of claims 1 to 9, or to be configured to execute a computer networking method according to any one of claims 10 to 18.

Images