System and method for routing data packets in unified wide area network

Abstract

Methods and networks for routing data packets in a unified wide area network (WAN) are provided. The method includes encapsulating, by an ingress aggregation router, a data packet, and forwarding the encapsulated data packet to an ingress backbone router. The encapsulated data packet includes a first tag. The ingress backbone router selects an optimized traffic engineering channel, replaces the first tag with the optimized traffic engineering channel, and forwards the encapsulated data packet along the optimized traffic engineering channel.

Description

Background Art

[0001] The mass commercialization of cloud computing has led to cloud providers offering private wide area networks (WANs). These initial deployments use two WANs - a software-defined WAN (also known as a "SWAN") to connect both the cloud's data center and the Internet peering edge to carry traffic within the data center and a standard-defined WAN (also known as a "CORE") for Internet traffic. Summary of the invention

[0002] In some implementations, the technology described herein relates to a method for routing data, comprising: obtaining an encapsulated data packet having a first label, wherein the first label is an egress site label, wherein the encapsulated data packet has a destination; selecting an optimized traffic engineering channel from two or more channels; replacing the first label with the selected optimized traffic engineering channel label; and forwarding the data packet along the selected optimized traffic engineering channel.

[0003] In some implementations, the technology described herein relates to a unified wide area network (WAN), comprising: a backbone router including a traffic engineering module, wherein the traffic engineering module sets a traffic engineering channel between the backbone router and a destination router for encapsulated data packets; and an aggregation router including a traffic steering module, wherein the traffic steering module encapsulates the encapsulated data packets and forwards the encapsulated data packets to the backbone router.

[0004] In some implementations, the technology described herein relates to a method for routing data packets by an ingress backbone router, comprising: receiving an encapsulated data packet having a first label and a second label, wherein the first label is an egress site label and the second label is a node segment identifier (node ​​SID); determining whether a service engineering channel to the egress site is available; when the service engineering channel to the egress site is available, replacing the first label with the service engineering channel and forwarding the encapsulated data packet along the service engineering channel; and when the service engineering channel to the egress site is not available, removing the first label and forwarding the encapsulated data packet using the node SID.

[0005] This Summary is provided to introduce some concepts that are further described below in the Detailed Description. This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used to help limit the scope of the claimed subject matter.

[0006] Additional features and advantages of the embodiments of the present disclosure will be set forth in the following description, and in part will be apparent from the description, or may be learned through the practice of these embodiments. The features and advantages of these embodiments may be realized and obtained by means of the tools and combinations specifically indicated in the appended claims. These and other features will become more fully apparent from the following description and the appended claims, or may be learned through the practice of these embodiments as set forth below. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to describe the manner in which the above and other features of the present disclosure may be obtained, a more specific description will be presented by reference to specific implementations thereof illustrated in the accompanying drawings. For better understanding, the same elements are represented by the same reference numerals in the various drawings. Although some of the drawings may be schematic or exaggerated representations of concepts, at least some of the drawings may be drawn to scale. Understanding that the drawings depict some example implementations, these implementations will be described and explained with additional specificity and detail through the use of the drawings, in which:

[0008] Figure 1 are examples of two Wide Area Networks (WANs), a Software Defined WAN (SWAN) that carries traffic within a data center, and a Standard Defined WAN (CORE) that carries Internet traffic.

[0009] Figure 2 is an example of a simplified view of a unified WAN with two data center regions and two edge sites in accordance with at least one embodiment.

[0010] Figure 3 In accordance with at least one embodiment, a method for routing data is presented.

[0011] Figure 4 In accordance with at least one embodiment, a method for routing data is presented.

[0012] Figure 5 In accordance with at least one embodiment, a method for routing data packets by an ingress backbone router is presented.

[0013] Figure 6 In accordance with at least one embodiment, a unified WAN is presented that includes aggregation routers and backbone routers. DETAILED DESCRIPTION

[0014] Operating two separate large-scale WANs (CORE and SWAN) can be complex and cost-inefficient. For example, because routers are designated for either intra-data center traffic routing or for Internet traffic routing, load balancing between the two is not possible. In addition, because the data center edge connects to both the SWAN and CORE routers, this dual WAN connectivity can result in wasteful use of expensive network equipment and limited power supplies. Building new data center regions and edge sites only makes this problem worse.

[0015] Segmented WAN architectures can make capacity planning difficult. At a given time, one WAN may be underutilized while another may be overutilized. Furthermore, it may become too expensive to obtain optimal capacity from both WANs in each geographic region and to build the required redundancy on each WAN. At the same time, Internet traffic has been growing steadily, and the Resource Reservation Protocol-Traffic Engineering (RSVP-TE) protocol used in the CORE network is reaching scale limitations due to the existing size of the CORE network topology. The SWAN routers run the Border Gateway Protocol (BGP), which is responsible for generating and updating routing table information and storing it in the router memory.

[0016] Therefore, there is a need for a unified wide area network (unified WAN) that can use software-defined control to carry both data center services and Internet services. In addition, new routing methods are needed to route data at a much larger scale than any one of the two WANs currently handles.

[0017] Figure 1 An example of a current environment with two wide area networks (WANs) is provided, a software defined WAN 101 (also referred to as a "SWAN") carrying intra-datacenter traffic 110 and a standard defined WAN 103 (also referred to as a "CORE") carrying Internet traffic 112. SWAN 101 includes a SWAN router 105, and CORE includes a CORE router 107. SWAN router 105 and CORE router 107 use completely different protocol stacks. One possible disadvantage of operating two independent WANs is that it may make capacity planning difficult. Another possible disadvantage is that current data center edges need to support both SWAN and CORE routers, resulting in wasteful use of expensive network equipment and limited power supplies.

[0018] By merging the SWAN and CORE networks into a unified WAN, new methods are needed to route data. Due to the existing size of the CORE network topology, the traffic engineering (RSVP-TE) protocol used in the CORE network is reaching scale limitations and therefore cannot be utilized in the unified WAN. The SWAN routers run the Border Gateway Protocol (BGP), which is responsible for generating and updating routing table information and storing it in the router memory. If BGP were used, the unified WAN routers would need to maintain the entire Internet routing table, which would be cost-prohibitive since the routing table would include millions of routes. One possible disadvantage of converting each router in the unified WAN to run BGP with a complete routing table is the need to have high ternary content addressable memory (TCAM) for all routers.

[0019] Therefore, unified WANs assign two roles to routers: (1) aggregation routers that maintain complete IP routing tables and (2) backbone routers that operate as forwarding-only nodes. Figure 2 An example of a simplified view of a unified WAN 200 with two data center regions 236 and two edge sites 238 is provided in accordance with at least one embodiment. One possible benefit of having only an aggregation router to hold a complete routing table is that it allows the remaining backbone routers to be simpler, cheaper forwarding-only devices with smaller ternary content addressable memories (TCAMs).

[0020] In some embodiments, an aggregation router (e.g. Figure 2 Aggregation routers 202_1 and 202_3 in the data center may be connected to data center edge routers, such as data center edge router 206_1 and data center edge router 206_2, respectively. A data center edge router is a dedicated router located at the network boundary of a data center that allows interconnection and exchange of routing data with aggregation routers for the purpose of interconnection with other data centers and the Internet.

[0021] In some embodiments, aggregation routers in the unified WAN (e.g., aggregation router 202_2 and aggregation router 202_4) can be connected to peer edge 208_1 and peer edge 208_2, respectively. Peer edge allows interconnection and exchange of Internet routing data between automation systems, where an automation system is a network or group of networks managed by a single routing policy. Internet peer routers exchange routing data between networks managed by different entities.

[0022] The unified WAN 200 includes a plurality of aggregation routers 202_1, 202_2, 202_3, and 202_4 and a plurality of backbone routers 204_1, 204_2, 204_3, and 204_4. In some embodiments, the plurality of aggregation routers and the plurality of backbone routers are configured to be used in combination with Figure 4 The method further discussed routes both intra-datacenter data packets and Internet data packets.

[0023] In some embodiments, the unified WAN 200 can route data packets between two data centers. For example, the first data center edge router 206_1 can use a combination of Figure 4 As further discussed and as shown by arrow 210, the data packet is sent to the second data center edge router 206_2 via the ingress aggregation router 202_1, the ingress backbone router 204_1, the egress backbone router 204_3, and the egress aggregation router 202_3. In some embodiments, the unified WAN 200 can route data packets between external networks and data centers. For example, the peer edge router 208_1 can use a method such as the one in combination with Figure 4 As further discussed and as shown by arrow 212, the method sends the data packet to the data center edge router 206_1 via the ingress aggregation router 202_2, the ingress backbone router 204_2, the egress backbone router 204_1, and the egress aggregation router 202_1. Figure 6 The roles of aggregation routers and backbone routers are discussed further.

[0024] Figure 3 A method 300 for routing data is presented in accordance with at least one embodiment. The method 300 includes receiving (e.g., obtaining) an encapsulated data packet having a first label at stage 317, wherein the first label is an egress site label, wherein the encapsulated data packet has a destination. In some embodiments, the method 300 also includes a method for routing data in a unified WAN. In some embodiments, the unified WAN is comprised of multiple sites, wherein each unified WAN site is assigned a static identifier referred to as a site label. For example, the site label can be a unique alphanumeric site label. One possible benefit of having a unique static identifier is that it simplifies the method of routing, as further explained below.

[0025] In some embodiments, each unified WAN site includes a plurality of aggregation routers and a plurality of backbone routers. In some embodiments, the plurality of aggregation routers and the plurality of backbone routers are configured to route data packets within the data center and Internet data packets.

[0026] In some embodiments, the encapsulated data packet is received by an ingress backbone router. In some embodiments, the encapsulated data packet is sent by an ingress aggregation router. In some embodiments, the ingress aggregation router encapsulates the encapsulated data packet, including adding a first label to the data packet. In some embodiments, the encapsulation is accomplished by multi-protocol label switching (MPLS). Other embodiments may encapsulate the data packet in Internet Protocol version 6, which provides equivalent functionality for the content outlined below. MPLS is a routing technology that guides data packets from one node to the next node based on a label stack rather than a network address. One possible advantage of MPLS is that it allows packet forwarding decisions to be made only on the contents of the label without examining the packet itself, as discussed further below. In some embodiments, MPLS may encapsulate packets of a network protocol. For example, MPLS may add an additional label to a packet header.

[0027] In some embodiments, the ingress aggregation router stores a complete IP routing table. For example, the complete routing table includes both the Internet routing table and the data center routing table. In some embodiments, the aggregation router includes the Border Gateway Protocol (BGP). BGP is a protocol that is the basis of the global routing system of the Internet. It manages how packets are routed from network to network by exchanging routing and reachability information between edge routers.

[0028] In some embodiments, BGP is responsible for generating and updating complete routing table information on the aggregation router. In some embodiments, BGP on the aggregation router receives routes announced by at least one of a BGP route reflector or a BGP client. In some embodiments, BGP selects one or more equal-cost BGP next hops for each prefix in the routing table based on the received routes. For example, the prefix is ​​an alphanumeric value of the destination address. In some embodiments, one or more BGP next hops are one or more aggregation routers at a unified WAN network egress site. In some embodiments, one or more BGP next hops are endpoints outside of a unified WAN network egress site in a legacy portion of the WAN. For example, if some WANs have been converted to a unified WAN, but some WANs still operate as SWANs or as CORE WANs, the destination of the data packet is in the legacy portion of the WAN (e.g., SWAN or CORE).

[0029] In some embodiments, the egress site label refers to the backbone egress site on the shortest path to the destination of the data packet. Figure 2, if the data packet received by the ingress aggregation router 202_1 has a destination at the second data center edge 206_2, the backbone egress site is the site where the egress backbone router 204_3 resides because the egress backbone router 204_3 has the shortest path to the destination of the data packet (data center edge router 206_2).

[0030] One possible benefit of encapsulating data packets with egress site labels is that the ingress backbone router can perform traffic engineering without IP routing.

[0031] In some embodiments, the encapsulated data packet further includes a second label. For example, the second label can be added by MPLS. In some embodiments, the second label is a node segment identifier (node ​​SID). In some embodiments, the node SID is a BGP next hop. The BGP next hop comes from a routing table generated by BGP.

[0032] In some embodiments, the ingress backbone router resides (e.g., is located) in the same site (e.g., on the same WAN site) as the ingress aggregation router. In some embodiments, the ingress backbone router does not maintain a full IP routing table. One possible benefit of having only aggregation routers to run BGP with a full routing table is that it allows the remaining backbone routers to be simpler, cheaper forwarding-only devices with smaller ternary content addressable memory (TCAM).

[0033] In some embodiments, the ingress aggregation routers are directly connected to the ingress backbone routers with equal capacity, but each ingress backbone router may not be an equal choice for the ingress aggregation router. For example, one ingress backbone router may have a longer path to the destination of the data packet, which may increase latency. In another example, the ingress backbone router may have less bandwidth available for the egress site, which may cause congestion.

[0034] In some embodiments, an ingress backbone router to which an ingress aggregation router forwards encapsulated data packets is selected based on a weighted business-oriented routing calculation. In some embodiments, the weighted business-oriented routing calculation is performed by a unified WAN agent running as a process on the ingress aggregation router. In some embodiments, the unified WAN agent uses an HTTPS server to communicate with the controller. In some embodiments, the controller calculates the weighted business-oriented routing calculation for the unified WAN agent. In some embodiments, the unified WAN agent programs the business-oriented routing on the ingress aggregation router based on the weighted business-oriented calculation of the controller. For example, the controller can exclude the ingress backbone router with the shortest path delay from the ingress aggregation router to the egress site, which shortest path delay exceeds the optimal delay threshold, and then the controller can calculate the weight using a single item maximum flow from the ingress aggregation router to the egress site.

[0035] Method 300 also includes selecting an optimized service engineering channel from two or more channels at stage 318. In some embodiments, the backbone router performs service engineering optimization. In some embodiments, service engineering optimization includes measuring service matrix (TM) and network diagram. Unified WAN TM is a collection of service trunks and bandwidth for each trunk. Service trunks are aggregated service flows from source backbone routers (e.g., inlet backbone routers) to destination sites for specific service categories. In some embodiments, four main service categories can exist in unified WAN: voice, interactive, best effort and scavenger. Network diagram is a dynamic topology composed of sites, nodes, links, other features or combinations thereof. For example, each node and link can have different attributes, including interface address, device role, link operating bandwidth, bandwidth reserved for RSVP-TE, link metric, whether links or nodes should be avoided due to maintenance activities, link reliability information, other attributes or combinations thereof.

[0036] In some embodiments, the business engineering optimization has two phases: a path calculation phase and an optimization phase. In the path calculation phase, online calculations of paths on a dynamic topology for all business trunks can be performed. In the optimization phase, a priority fairness optimization solver can assign business trunks to paths. TMs can be divided based on the business class of the trunk and / or each business class can be optimized differently. In some embodiments, the priority fairness solver links four solvers (maximum-minimum fairness, minimized cost, minimized maximum utilization, and distinct paths) in different combinations based on business class.

[0037] In some embodiments, the ingress backbone router further comprises a unified WAN agent. In some embodiments, the unified WAN agent programs two or more traffic engineering routes on the ingress backbone router based on traffic engineering optimization.

[0038] In some embodiments, selecting the optimized traffic engineering channel from the two or more channels further comprises the ingress backbone router using the egress site label to determine one or more traffic engineering channels for use between the ingress backbone router and an egress backbone router located at the egress site.

[0039] The method 300 includes replacing the first label with the selected traffic engineering channel label at stage 320. In some embodiments, there may be no operable uplink channels available, and therefore the traffic engineering channel may not be used, such as in conjunction with Figure 5 further discussed.

[0040] Method 300 includes forwarding data packets along the selected service engineering channel at stage 322. In some embodiments, the data packet is forwarded by the ingress backbone router along the selected service engineering channel to the egress backbone router. In some embodiments, the service engineering channel terminates at the egress backbone router. A possible benefit of terminating the service engineering channel at the egress backbone router rather than the egress aggregation router is that the node SID label must be removed before the data packet is delivered to the intended destination. It is not easy for the router to support popping (e.g., removing) the label stack, so at least one segment routing jump (e.g., from the egress backbone router to the egress aggregation router) is required to remove the node SID label. In some embodiments, the segment routing implementation on the provider router only allows the second-to-last jump out (PHP), which means that the second-to-last router (e.g., the egress backbone router) must remove the node SID label and then forward the data packet to the final router (e.g., the egress aggregation router). In some embodiments, the egress backbone router uses the second label (node ​​SID) to perform segment routing and remove the node SID label. For example, the egress backbone router uses the node SID label to segment route the data packet to the final destination on the egress aggregation router.

[0041] In some embodiments, the traffic engineering channel terminates at the egress aggregation router. Termination at the final router will require support for the final jump out. In the final jump out, the node SID label can be removed at the final router (e.g., the egress aggregation router). In some embodiments, the egress aggregation router removes the second label from the encapsulated data packet.

[0042] Figure 4A method 400 for routing data is presented in accordance with at least one embodiment. In some embodiments, the method 400 also includes a method for routing data in a unified WAN. In some embodiments, the unified WAN is comprised of a plurality of sites, wherein each unified WAN site is assigned a static identifier referred to as a site tag. For example, the site tag may be a unique alphanumeric site tag. One possible benefit of having a unique static identifier is to simplify the method of routing, as further explained below.

[0043] In some embodiments, each unified WAN site includes multiple aggregation routers and multiple backbone routers. In some embodiments, the multiple aggregation routers and multiple backbone routers are configured to route both intra-data center data packets and Internet data packets.

[0044] Method 400 includes encapsulating (e.g., via an ingress aggregation router) a data packet having a destination. Data encapsulation includes adding a first label to the data packet at stage 414. In some embodiments, encapsulation is accomplished via Multi-Protocol Label Switching (MPLS). Other embodiments may encapsulate the data packet in Internet Protocol version 6, which provides equivalent functionality for the content outlined below. MPLS is a routing technology that directs data packets from one node to the next node based on labels rather than network addresses. One possible advantage of MPLS is that it allows packet forwarding decisions to be made only on the contents of the label without examining the packet itself, as discussed further below. In some embodiments, MPLS may encapsulate packets of a network protocol. For example, MPLS may add an additional label to a packet header.

[0045] In some embodiments, the ingress aggregation router stores a complete IP routing table. For example, the complete routing table includes both the Internet and the data center routing table. In some embodiments, the aggregation router includes the Border Gateway Protocol (BGP). BGP is a protocol that is the basis of the global routing system of the Internet. It manages how packets are routed from network to network by exchanging routing and reachability information between edge routers.

[0046] In some embodiments, BGP is responsible for generating and updating complete routing table information on the aggregation router. In some embodiments, BGP on the aggregation router receives routes published by at least one of a BGP route reflector or a BGP client. In some embodiments, BGP selects one or more equal-cost BGP next hops for each prefix in the routing table based on the received routes. For example, the prefix is ​​an alphanumeric value of the destination address. In some embodiments, one or more BGP next hops are one or more aggregation routers at a unified WAN network egress site. In some embodiments, one or more BGP next hops are endpoints outside the unified WAN network egress site in the legacy portion of the WAN. For example, if some WANs have been converted to a unified WAN, but some WANs still work as SWANs or as CORE WANs, the destination of the data packet is in the legacy portion of the WAN (e.g., SWAN or CORE).

[0047] In some embodiments, the first label is an egress site label. For example, the egress site label may refer to a backbone egress site on the shortest path to the destination of the data packet. Figure 2 , if the data packet received by the ingress aggregation router 202_1 has a destination at the second data center edge 206_2, the backbone egress site is the site where the egress backbone router 204_3 resides because the egress backbone router 204_3 has the shortest path to the destination of the data packet (data center edge router 206_2).

[0048] One possible benefit of encapsulating data packets with egress site labels is that the ingress backbone router can perform traffic engineering without IP routing.

[0049] In some embodiments, encapsulating the data packet further comprises adding a second label to the data packet. For example, the second label may be added by MPLS. In some embodiments, the second label is a node segment identifier (node ​​SID). In some embodiments, the node SID is a BGP next hop. The BGP next hop comes from a routing table generated by BGP.

[0050] The method 400 also includes forwarding the encapsulated data packet to an ingress backbone router at stage 416. In some embodiments, the ingress backbone router resides at (e.g., is located at) the same site as the ingress aggregation router. In some embodiments, the ingress backbone router does not store a complete IP routing table. One possible benefit of having only aggregation routers to run BGP with a complete routing table is that it allows the remaining backbone routers to be simpler, cheaper forwarding-only devices with smaller ternary content addressable memories (TCAMs).

[0051] In some embodiments, the ingress aggregation routers are directly connected to the ingress backbone routers with equal capacity, but each ingress backbone router may not be an equal choice for the ingress aggregation router. For example, one ingress backbone router may have a longer path to the destination of the data packet, which may increase latency. In another example, the ingress backbone router may have less bandwidth available for the egress site, which may cause congestion.

[0052] In some embodiments, an ingress backbone router to which an ingress aggregation router forwards encapsulated data packets is selected based on a weighted business-oriented routing calculation. In some embodiments, the weighted business-oriented routing calculation is performed by a unified WAN agent running as a process on the ingress aggregation router. In some embodiments, the unified WAN agent uses an HTTPS server to communicate with the controller. In some embodiments, the controller calculates the weighted business-oriented routing calculation for the unified WAN agent. In some embodiments, the unified WAN agent programs the business-oriented routing on the ingress aggregation router based on the weighted business-oriented calculation of the controller. For example, the controller can exclude the ingress backbone router with the shortest path delay from the ingress aggregation router to the egress site, which shortest path delay exceeds the optimal delay threshold, and then the controller can calculate the weight using a single item maximum flow from the ingress aggregation router to the egress site.

[0053] Method 400 also includes selecting optimized business engineering channels from two or more channels at stage 418. In some embodiments, backbone routers perform business engineering optimization. In some embodiments, business engineering optimization includes measuring business matrix (TM) and network diagram. Unified WAN TM is a collection of business trunks and bandwidth for each trunk. Business trunks are aggregated business flows from source backbone routers (e.g., inlet backbone routers) to destination sites for specific business categories. In some embodiments, four main business categories can exist in unified WAN: voice, interactive, best effort and scavenger. Network diagram is a dynamic topology composed of sites, nodes, links, other features or combinations thereof. For example, each node and link can have different attributes, including interface address, device role, link operation bandwidth, bandwidth reserved for RSVP-TE, link metric, whether links or nodes should be avoided due to maintenance activities, link reliability information, other attributes or combinations thereof.

[0054] In some embodiments, the business engineering optimization has two phases: a path calculation phase and an optimization phase. In the path calculation phase, online calculations of paths on a dynamic topology for all business trunks can be performed. In the optimization phase, a priority fairness optimization solver can assign business trunks to paths. TMs can be divided based on the business class of the trunk and / or each business class can be optimized differently. In some embodiments, the priority fairness solver links four solvers (maximum-minimum fairness, minimized cost, minimized maximum utilization, and distinct paths) in different combinations based on business class.

[0055] In some embodiments, the ingress backbone router further comprises a unified WAN agent. In some embodiments, the unified WAN agent programs two or more traffic engineering routes on the ingress backbone router based on traffic engineering optimization.

[0056] In some embodiments, selecting the optimized traffic engineering channel from the two or more channels further comprises the ingress backbone router using the egress site label to determine one or more traffic engineering channels for use between the ingress backbone router and an egress backbone router located at the egress site.

[0057] Method 400 includes replacing the first tag with the selected traffic engineering channel tag at stage 420. In some embodiments, there may be no operable uplink channels available, and therefore the traffic engineering channel may not be used, such as in conjunction with Figure 5 further discussed.

[0058] Method 400 includes forwarding data packets along the selected service engineering channel at stage 422. In some embodiments, the data packet is forwarded by the ingress backbone router along the selected service engineering channel to the egress backbone router. In some embodiments, the service engineering channel terminates at the egress backbone router. A possible benefit of terminating the service engineering channel at the egress backbone router rather than the egress aggregation router is that the node SID label must be removed before the data packet is delivered to the intended destination. It is not easy for the router to support popping (e.g., removing) the label stack, so at least one segment routing jump (e.g., from the egress backbone router to the egress aggregation router) is required to remove the node SID label. In some embodiments, the segment routing implementation on the supplier router only allows the second to last jump out (PHP), which means that the second to last router (e.g., the egress backbone router) must remove the node SID label and then forward the data packet to the final router (e.g., the egress aggregation router). In some embodiments, the egress backbone router uses the second label (node ​​SID) to perform segment routing and remove the node SID label. For example, the egress backbone router uses the node SID label to segment route the data packet to the final destination on the egress aggregation router.

[0059] In some embodiments, the traffic engineering channel terminates at the egress aggregation router. Termination at the final router will require support for the final jump out. In the final jump out, the node SID label can be removed at the final router (e.g., the egress aggregation router). In some embodiments, the egress aggregation router removes the second label from the encapsulated data packet.

[0060] Figure 5 According to at least one embodiment, a method 500 for routing data by an ingress backbone router in the absence of an available channel is presented. The method 500 includes receiving (e.g., obtaining) an encapsulated data packet having a first label and a second label at stage 524, wherein the first label is an egress site label and the second label is a node segment identifier (node ​​SID) label. In some embodiments, the ingress backbone router receives the encapsulated data packet from an ingress aggregation router. For example, as previously described in connection with Figure 4 As discussed, the ingress aggregation router can encapsulate the data packet. In some embodiments, the ingress aggregation router encapsulates the data packet with Multi-Protocol Label Switching (MPLS).

[0061] The method 500 also includes determining whether a traffic engineering channel to the egress site is available at stage 526. At stage 528, when the traffic engineering channel to the egress site is available, the ingress backbone router replaces the first label with the traffic engineering channel label and forwards the data packet along the traffic engineering channel. At stage 530, when the traffic engineering channel to the egress site is not available, the ingress backbone router removes the first label and forwards the data packet using the node SID. In some embodiments, the node SID is a BGP next hop from a routing table stored in an ingress aggregation router.

[0062] In some embodiments, if no traffic engineering channel is active for an egress site due to, for example, a failure, then the second label (e.g., node SID) is used as a fallback. For example, when an ingress backbone router does not have an operational upstream traffic engineering channel to a particular egress site, the unified WAN agent on the ingress backbone router removes the first label (e.g., egress site label) and forwards the data packet using segment routing for the BGP next-hop node SID. One possible advantage of doing this is that failures in the network are handled quickly and transparently by the routers without immediate intervention by the controller.

[0063] In some embodiments, when the ingress backbone router has an operational upstream traffic engineering channel to a particular egress site, the unified WAN agent on the ingress backbone router replaces the first label (e.g., the egress site label) with the traffic engineering channel label and forwards the data packet along the traffic engineering channel.

[0064] Figure 6 According to at least one embodiment, a unified WAN 600 is shown that includes an aggregation router 602 and a backbone router 604. The aggregation router 602 includes a traffic steering module 632, wherein the traffic steering module 632 encapsulates data packets and forwards the encapsulated data packets to the backbone router 604. The backbone router 604 includes a traffic engineering module 634, wherein the traffic engineering module 634 sets a traffic engineering channel between the backbone router 604 and a destination router 636 for the data packets.

[0065] In some embodiments, unified WAN 600 is composed of multiple sites, where each unified WAN site is assigned a static identifier called a site tag. Figure 6Four different sites are included: Site A 640, Site B 642, Site C 644, and Site D 646, but it should be understood that in some embodiments there may be more or less than four sites. One possible benefit of having unique static identifiers is to simplify the method of routing, as further explained below. Aggregation router 602 and backbone router 604 reside at the same site, Site A 640, as shown by Figure 6 as displayed.

[0066] In some embodiments, the aggregation router 602 runs the Border Gateway Protocol (BGP). In some embodiments, BGP generates a complete IP routing table on the aggregation router. For example, the complete IP routing table includes both the Internet and the data center routing table. In some embodiments, the BGP on the aggregation router can receive routes announced by at least one of the BGP route reflector or the BGP client. In some embodiments, BGP selects one or more equal cost BGP next hops for each prefix in the routing table based on the received routes. In some embodiments, the one or more equal cost BGP next hops are egress aggregation routers. In some embodiments, the one or more equal cost BGP next hops are outside the egress aggregation router.

[0067] In some embodiments, each unified WAN site includes multiple aggregation routers and multiple backbone routers. In some embodiments, the multiple aggregation routers and multiple backbone routers are configured to route both intra-data center data packets and Internet data packets.

[0068] Figure 6 Only one backbone router 604 and only one aggregation router 602 on site A 640 are shown, but it should be noted that there may be more than one backbone router and more than one aggregation router on site A. The backbone router 604 to which the data packet is forwarded is selected by the business steering module 632 based on the weighted business steering route calculation. In some embodiments, the weighted business steering route calculation is performed by a unified WAN agent running as a process on the aggregation router 602. In some embodiments, the unified WAN agent uses an HTTPS server to communicate with the controller. In some embodiments, the controller calculates the weighted business steering route calculation for the unified WAN agent. In some embodiments, the unified WAN agent programs the business steering route on the aggregation router 602 based on the weighted business steering route calculation of the controller. In some embodiments, the controller excludes the backbone routers with the shortest path delay from the aggregation router to the destination site that exceeds the optimal delay threshold, and then uses the single item maximum flow from the aggregation router to the destination site to calculate the weight.

[0069] In some embodiments, the traffic steering module 632 encapsulating the data packet having the destination further comprises adding a first label to the data packet. In some embodiments, the encapsulation is accomplished via multi-protocol label switching (MPLS). In some embodiments, the first label is an egress site label. In some embodiments, the egress site label refers to a backbone egress site on the shortest path to the destination of the data packet. For example, in Figure 6 , the egress site label is site D 646 because the data packet's destination router 636 is there.

[0070] In some embodiments, the traffic steering module 632 encapsulating the data packet with the destination also includes adding a second label to the data packet. In some embodiments, the second label is a node segment identifier (node ​​SID). In some embodiments, the node SID is a BGP next hop as discussed above.

[0071] In some embodiments, backbone router 604 acts as a forwarding-only node that does not run BGP and does not maintain a complete IP routing table. A possible benefit of maintaining a complete IP routing table only at an aggregation router is that the size of the routing table can be smaller and therefore easier to manage at backbone router 604.

[0072] In some embodiments, the business engineering module 634 selects an optimized business engineering channel from two or more channels by performing business engineering optimization. In some embodiments, business engineering optimization includes measuring business matrix TM and network diagram. In some embodiments, TM is a collection of business trunks and bandwidth for each trunk. Business trunks are aggregated business flows from source backbone routers to destination sites for specific business categories. In some embodiments, four main business categories can exist in a unified WAN: voice, interactive, best effort and scavenger. The network diagram is a dynamic topology consisting of sites, nodes and links. For example, each node and link can have dozens of different attributes, including interface address, device role, link operation bandwidth, bandwidth reserved for RSVP-TE, link metrics, whether links or nodes should be avoided due to maintenance activities, and link reliability information.

[0073] In some embodiments, the business engineering optimization has two phases: a route calculation phase and an optimization phase. In the route calculation phase, online calculations of paths on a dynamic topology for all business trunks are performed. In the optimization phase, a priority fairness optimization solver assigns business trunks to paths. The TM is divided based on the business class of the trunks, and each business class is optimized differently. In some embodiments, the priority fairness optimization solver links two or more solvers in different combinations based on business class (max-min fairness, minimize cost, minimize maximum utilization, and distinct paths).

[0074] In some embodiments, the backbone router 604 further comprises a unified WAN agent. In some embodiments, the unified WAN agent programs two or more traffic engineering routes on the backbone router based on the traffic engineering optimization module.

[0075] In some embodiments, selecting the optimized traffic engineering path from the two or more paths further includes the backbone router 604 using the egress site label (e.g., site D 646) to determine one or more traffic engineering paths for use between the backbone router 604 and a destination router 636, wherein the destination router 636 is located at the egress site 646. For example, in Figure 6 In FIG. 6 , the two traffic engineering channels are a direct channel 648 from the backbone router 604 at site A 640 to the destination router 636 at site D 646 , and a second channel 650 through site B 642 and site C 644 .

[0076] In some embodiments, the backbone router 604 replaces the first label with the selected traffic engineering channel label. In some embodiments, there may be no active channel, and therefore no traffic engineering channel can be used, as previously described in conjunction with Figure 5 discussed.

[0077] In some embodiments, the backbone router 604 forwards the data packet along the selected traffic engineering channel to the destination router. In some embodiments, the traffic engineering channel terminates at the egress backbone router. Figure 6 In the embodiment of the present invention, the destination router 636 can be an egress backbone router. One possible benefit of terminating the service engineering channel at the egress backbone router rather than the egress aggregation router is that the node SID label must be removed before the data packet is delivered to the intended destination. It is not easy for the router to support popping the label stack, so at least one segment routing hop (e.g., from the egress backbone router to the egress aggregation router) is required to remove the node SID label. In some embodiments, the segment routing implementation on the provider router only allows the second-to-last hop out (PHP), which means that the second-to-last router (e.g., the egress backbone router) must remove the node SID and then forward the data packet to the final router (e.g., the egress aggregation router). In some embodiments, the egress backbone router uses the second label (node ​​SID) to perform segment routing and remove the node SID label. For example, the egress backbone router uses the node SID to segment route the data packet to the final destination on the egress aggregation router.

[0078] In some embodiments, the traffic engineering channel terminates at the egress aggregation router. Figure 6In the embodiment of the present invention, the destination router 636 may be an egress aggregation router. Termination at the final router will require support for the final jump out. In the final jump out, the node SID label may be removed at the final router (e.g., the egress aggregation router). In some embodiments, the egress aggregation router removes the second label from the encapsulated data packet.

[0079] The following are parts of embodiments according to the present disclosure:

[0080] A1. A method for routing data, comprising:

[0081] encapsulating, by an ingress aggregation router, a data packet having a destination, wherein the encapsulating includes adding a first label to the data packet;

[0082] forwarding the encapsulated data packet to an ingress backbone router;

[0083] selecting an optimized traffic engineering channel from two or more channels;

[0084] The first label is replaced with the selected optimized traffic engineering channel label; and the data packet is forwarded along the selected optimized traffic engineering channel.

[0085] A2. The method according to part A1, wherein the encapsulation is accomplished via Multi-Protocol Label Switching (MPLS).

[0086] A3. A method according to any of parts A1-A2, wherein the first tag is an exit site tag.

[0087] A4. The method of section A3, wherein the egress site label refers to a backbone egress site on a shortest path to a destination of the data packet.

[0088] A5. A method according to any of parts A1-A4, wherein encapsulating the data packet also includes adding a second label to the data packet.

[0089] A6. The method according to section A5, wherein the data packet is forwarded to an egress backbone router.

[0090] A7. A method according to part A5, wherein the second label is a node segment identifier (node ​​SID).

[0091] A8. The method according to section A7, wherein the node SID is a BGP next hop.

[0092] A9. A method according to any part of parts A1-A8, wherein the method also includes routing data in a unified wide area network (WAN).

[0093] A10. The method according to part A9, wherein the unified WA includes multiple sites.

[0094] A11. A method according to part A10, wherein each of the plurality of sites is assigned a site label.

[0095] A12. The method according to section A10, wherein each of the plurality of sites comprises a plurality of aggregation routers and a plurality of backbone routers.

[0096] A13. The method of section A12, wherein the plurality of aggregation routers and the plurality of backbone routers are configured to route both intra-datacenter data packets and Internet data packets.

[0097] A14. A method according to any of parts A1-A13, wherein the ingress aggregation router maintains a complete IP routing table.

[0098] A15. A method according to part A14, wherein the complete IP routing table includes both an Internet routing table and a data center routing table.

[0099] A16. A method according to any part of parts A14 and A15, wherein the full IP table is generated by the Border Gateway Protocol (BGP).

[0100] A17. The method of section A16, wherein the BGP on the ingress aggregation router receives routes announced by at least one of a BGP route reflector or a BGP client.

[0101] A18. The method of section A17, wherein the BGP selects one or more equal-cost BGP next hops for each prefix in the routing table based on the received routes.

[0102] A19. The method of section A18, wherein the one or more equal-cost BGP next hops are egress aggregation routers.

[0103] A20. A method as described in section A18, wherein the one or more equal-cost BGP next hops are endpoints other than an egress site in a legacy portion of the WAN.

[0104] A21. The method of section A10, wherein the ingress aggregation router and the ingress backbone router reside at the same WAN site.

[0105] A22. The method of any of sections A3-A21, wherein the ingress backbone router to which the encapsulated data packet is forwarded is selected based on a weighted traffic-directed routing calculation by the ingress aggregation router.

[0106] A23. A method according to section A22, wherein the weighted traffic-oriented routing calculation is performed by a unified WAN agent running as a process on the ingress aggregation router.

[0107] A24. A method according to part A23, wherein the unified WAN agent uses an HTTPS server to communicate with the controller.

[0108] A25. A method according to part A24, wherein the controller calculates weighted business-oriented routing calculations for the unified WAN agent.

[0109] A26. The method of section A25, wherein the unified WAN agent programs the weighted service-oriented routing on the ingress aggregation router based on the weighted service-oriented routing calculation of the controller.

[0110] A27. A method according to section A26, wherein the controller excludes the ingress backbone router with the shortest path delay from the ingress aggregation router to the egress site that exceeds the optimal delay threshold, and then calculates the weight using the single item maximum flow from the ingress aggregation router to the egress site.

[0111] A28. A method according to any part of parts A1-A27, wherein the ingress backbone router does not store a complete IP routing table.

[0112] A29. A method according to any of parts A3-A28, wherein selecting an optimized traffic engineering channel from two or more channels also includes an ingress backbone router performing traffic engineering optimization.

[0113] A30. A method according to part A29, wherein the traffic engineering optimization includes measuring the traffic matrix TM and the network diagram.

[0114] A31. A method according to section A30, wherein the service matrix is ​​a collection of service trunks and the bandwidth for each trunk.

[0115] A32. A method according to section A31, wherein the service trunk is an aggregated service flow from an ingress backbone router to a destination site for a service class.

[0116] A33. A method according to section A32, wherein the service category is one of voice, interactive, best effort, and scavenger.

[0117] A34. A method according to section A30, wherein the network graph is a dynamic topology consisting of sites, nodes, and links.

[0118] A35. A method according to part A34, wherein the nodes and links have one or more attributes.

[0119] A36. A method according to section A35, wherein the one or more attributes are one of an interface address, a device role, a link operating bandwidth, a bandwidth reserved for RSVP-TE, a link metric, information about whether a link or node should be avoided, and link reliability information.

[0120] A37. A method according to any part of parts A29-A36, wherein the business engineering optimization includes a path calculation phase and an optimization phase.

[0121] A38. A method according to section A37, wherein in the path calculation phase, online calculation of paths on the dynamic topology is performed for all service trunks.

[0122] A39. A method according to part A38, wherein in the optimization phase, the priority fairness optimization solver assigns service trunks to paths.

[0123] A40. A method according to section A38, wherein each service trunk is optimized differently.

[0124] A41. A method according to section A39, wherein the priority fairness optimization solver also includes linking four solvers in different combinations based on business categories.

[0125] A42. A method according to part A41, wherein the four solvers include a maximum-minimum fairness solver, a minimum cost solver, a minimum maximum utilization solver and a different path solver.

[0126] A43. A method according to any part of parts A29-A42, wherein the ingress backbone router also includes a unified WAN agent.

[0127] A44. The method according to A43, wherein the unified WAN agent further comprises programming two or more traffic engineering channels on the ingress backbone router based on traffic engineering optimization.

[0128] A45. A method according to any part of parts A3-A44, wherein selecting an optimized traffic engineering channel from two or more channels also includes an ingress backbone router using an egress site label to determine one or more channels between the ingress backbone router and an egress backbone router, wherein the egress backbone router is located at the egress site.

[0129] A46. The method according to section A45, wherein forwarding the data packet along the traffic engineering channel further comprises the ingress backbone router using the selected traffic engineering channel to forward the data packet to the egress backbone router.

[0130] A47. A method according to section A46, wherein the service engineering channel terminates at an egress backbone router.

[0131] A48. The method of section A47, wherein the egress backbone router uses the second label to perform segment routing.

[0132] A49. The method of section A48, wherein the egress backbone router removes the second label from the encapsulated data packet.

[0133] A50. A method according to section A46, wherein the traffic engineering channel terminates at an egress aggregation router.

[0134] A51. The method of section A50, wherein the egress aggregation router removes the second label from the encapsulated data packet.

[0135] B1. A unified wide area network (WAN), comprising:

[0136] an aggregation router, comprising a traffic steering module, wherein the traffic steering module encapsulates data packets and forwards the encapsulated data packets to a backbone router; and

[0137] A backbone router comprises a traffic engineering module, wherein the traffic engineering module sets a traffic engineering channel between the backbone router and a destination router for encapsulated data packets.

[0138] B2. A unified WAN according to section B1, wherein the unified WAN includes multiple sites.

[0139] B3. A unified WAN as described in section B2, wherein each site has an assigned static identifier called a site tag.

[0140] B4. A unified WAN as described in section B1, wherein the aggregation router and the backbone router reside at the same site.

[0141] B5. A unified WAN as described in any of sections B1-B4, wherein the aggregation router runs Border Gateway Protocol (BGP).

[0142] B6. A unified WAN as described in section B5, wherein the BGP generates a complete IP routing table on the aggregation router.

[0143] B7. A unified WAN according to section B6, wherein the complete IP routing table includes both an Internet routing table and a data center routing table.

[0144] B8. A unified WAN as described in any of sections B5-B6, wherein the BGP on the aggregation router receives routes announced by at least one of a BGP route reflector or a BGP client.

[0145] B9. A unified WAN as described in any of sections B5-B8, wherein the BGP selects one or more equal-cost BGP next hops for each prefix in the routing table based on the received routes.

[0146] B10. A unified WAN as described in section B9, wherein the one or more equal-cost BGP next hops are egress aggregation routers.

[0147] B11. A unified WAN as described in section B9, wherein the one or more equal-cost BGP next hops are outside the egress aggregation router.

[0148] B12. A unified WAN according to any of sections B2-B11, wherein each unified WAN site includes multiple aggregation routers and multiple backbone routers.

[0149] B13. A unified WAN according to any of sections B2-B12, wherein the plurality of aggregation routers and the plurality of backbone routers are configured to route both intra-data center data packets and Internet data packets.

[0150] B14. A unified WAN according to any of parts B1-B13, wherein the backbone router to which the encapsulated data packet is forwarded is selected by the traffic steering module based on weighted traffic steering routing calculation.

[0151] B15. A unified WAN according to section B14, wherein the weighted service-oriented routing calculation is performed by a unified WAN agent.

[0152] B16. A unified WAN as described in section B15, wherein the unified WAN agent runs as a process on the aggregation router.

[0153] B17. A unified WAN according to any of sections B15-B16, wherein the unified WAN agent uses an HTTPS server to communicate with the controller.

[0154] B18. A unified WAN according to section B17, wherein the controller calculates weighted business-oriented routing calculations for the unified WAN agents.

[0155] B19. A unified WAN according to section B18, wherein the unified WAN agent further comprises programming the business-oriented routing on the aggregation router based on the weighted business-oriented routing calculation of the controller.

[0156] B20. A unified WAN as described in section B19, wherein the controller excludes backbone routers with shortest path delays from the aggregation router to the destination site that exceed an optimal delay threshold, and then calculates the weight using a single item maximum flow from the aggregation router to the destination site.

[0157] B21. A unified WAN according to any of parts B1-B20, wherein the business steering module encapsulating the data packet having a destination further includes adding a first label to the data packet.

[0158] B22. A unified WAN according to section B21, wherein the encapsulation is accomplished via Multi-Protocol Label Switching (MPLS).

[0159] B23. A unified WAN according to any of sections B21-B22, wherein the first tag is an egress site tag.

[0160] B24. A unified WAN according to section B23, wherein the egress site label refers to a backbone egress site on the shortest path to the destination of the data packet.

[0161] B25. A unified WAN according to any of parts B1-B24, wherein the traffic steering module encapsulating the data packet having the destination further includes adding a second label to the data packet.

[0162] B26. A unified WAN according to section B25, wherein the second label is a node segment identifier (node ​​SID).

[0163] B27. A unified WAN according to section B26, wherein the node SID is a BGP next hop.

[0164] B28. A unified WAN according to any of sections B1-B27, wherein the backbone router does not store a complete IP routing table.

[0165] B29. A unified WAN according to any of parts B1-B28, wherein the traffic engineering module selects an optimized traffic engineering channel from two or more channels by performing traffic engineering optimization.

[0166] B30. A unified WAN according to section B29, wherein the traffic engineering optimization includes measuring a traffic matrix (TM) and a network map.

[0167] B31. A unified WAN according to section B30, wherein the TM is a collection of service trunks and the bandwidth for each trunk.

[0168] B32. A unified WAN according to section B31, wherein the service trunk is an aggregated service flow from a source backbone router to a destination site for a specific service class.

[0169] B33. A unified WAN according to section B32, wherein the traffic class is one of voice, interactive, best effort, and scavenger.

[0170] B34. A unified WAN according to any of sections B30-B33, wherein the network graph is a dynamic topology consisting of sites, nodes, and links.

[0171] B35. A unified WAN according to section B34, wherein the nodes and links have one or more attributes.

[0172] B36. A unified WAN according to section B35, wherein the one or more attributes include at least one of an interface address, a device role, a link operating bandwidth, a bandwidth reserved for RSVP-TE, a link metric, information about whether a link or node should be avoided due to maintenance activity, and link reliability information.

[0173] B37. A unified WAN according to any of sections B29-B36, wherein the traffic engineering optimization includes a path calculation phase and an optimization phase.

[0174] B38. A unified WAN according to section B37, wherein during the path calculation phase, it also includes performing online calculation of paths on a dynamic topology for all service trunks.

[0175] B39. A unified WAN according to any of sections B37-B38, wherein in the optimization phase, the priority fairness optimization solver assigns service trunks to paths.

[0176] B40. A unified WAN according to section B39, wherein the priority fairness optimization solver also includes linking two or more solvers in different combinations based on traffic class.

[0177] B41. A unified WAN according to section B40, wherein the two or more solvers include at least two of a max-min fairness solver, a minimize cost solver, a minimize maximum utilization solver, and a distinct path solver.

[0178] B42. A unified WAN according to any of parts B1-B41, wherein the backbone router also includes a unified WAN agent.

[0179] B43. A unified WAN according to section B42, wherein the unified WAN agent further comprises programming two or more traffic engineering routes on a backbone router on a traffic engineering module.

[0180] B44. A unified WAN according to any of sections B29-B43, wherein selecting an optimized traffic engineering channel from two or more channels also includes a backbone router using an exit site label to determine one or more traffic engineering channels for use between the backbone router and a destination router, wherein the destination router is located at the exit site.

[0181] B45. A unified WAN according to section B44, wherein the backbone router replaces the egress site label with the selected service engineering channel label.

[0182] B46. A unified WAN according to section B45, wherein the encapsulated data packets are forwarded along the selected traffic engineering channel to the destination router.

[0183] B47. A unified WAN as described in section B46, wherein the service engineering channel terminates at an egress backbone router.

[0184] B48. A unified WAN according to section B47, wherein the egress backbone router uses the second label to perform segment routing.

[0185] B49. A unified WAN according to section B48, wherein the egress backbone router removes the second label from the encapsulated data packet.

[0186] B50. A unified WAN according to any of sections B46-B49, wherein the traffic engineering channel terminates at an egress aggregation router.

[0187] B51. A unified WAN according to section B50, wherein the egress aggregation router removes the second label from the encapsulated data packet.

[0188] C1. A method for routing data packets by an ingress backbone router, comprising:

[0189] receiving (e.g., obtaining) an encapsulated data packet having a first label and a second label, wherein the first label is an egress site label and the second label is a node segment identifier (node ​​SID);

[0190] Determine whether the business engineering channel to the egress site is available;

[0191] When a traffic engineering channel to the egress site is available, replacing the first label with the traffic engineering channel label and forwarding the encapsulated data packet along the traffic engineering channel; and

[0192] When the traffic engineering channel to the egress site is not available, the first label is removed and the encapsulated data packet is forwarded using the node SID.

[0193] C2. The method of section C1, wherein the encapsulated data packet is received from an ingress aggregation router.

[0194] C3. The method according to section C2, wherein the node SID is a BGP next hop.

[0195] C4. The method of section C3, wherein the BGP next hop is from a routing table stored in an ingress aggregation router.

[0196] C5. A method according to any of parts C2-C4, wherein the encapsulated data packet is encapsulated using Multi-Protocol Label Switching (MPLS).

[0197] C6. A method according to any of parts C1-C5, wherein the ingress backbone router also includes a unified WAN agent.

[0198] C7. The method of section C6, wherein when the traffic engineering channel is unavailable, the unified WAN agent removes the first label, and when the traffic engineering channel is available, the unified WAN agent replaces the first label with the traffic engineering channel.

[0199] D1. A method for routing data, comprising:

[0200] receiving an encapsulated data packet having a first label, wherein the first label is an egress site label, wherein the encapsulated data packet has a destination;

[0201] selecting an optimized traffic engineering channel from two or more channels;

[0202] replacing the first label with the selected optimized traffic engineering channel; and forwarding the encapsulated data packet along the optimized traffic engineering channel.

[0203] D2. The method of section D1, wherein the encapsulated data packet is received by an ingress backbone router from an ingress aggregation router.

[0204] D3. A method according to any of parts D1-D2, wherein the encapsulation is accomplished via Multi-Protocol Label Switching (MPLS).

[0205] D4. The method of any of sections D1-D3, wherein the egress site label refers to a backbone egress site on a shortest path to a destination of the encapsulated data packet.

[0206] D5. A method according to any of parts D1-D4, wherein the encapsulated data packet also includes a second tag.

[0207] D6. The method according to section D5, wherein the encapsulated data packet is forwarded to an egress backbone router.

[0208] D7. A method according to part D5, wherein the second label is a node segment identifier (node ​​SID).

[0209] D8. The method according to D7, wherein the node SID is the BGP next hop.

[0210] D9. A method according to any part of parts D2-D8, wherein the method also includes routing data in a unified wide area network (WAN).

[0211] D10. A method according to section D9, wherein the unified WAN consists of multiple sites.

[0212] D11. A method according to section D10, wherein each unified WAN site is assigned a site label.

[0213] D12. The method according to any of sections D10-D11, wherein each of the plurality of sites comprises a plurality of aggregation routers and a plurality of backbone routers.

[0214] D13. The method of section D12, wherein the plurality of aggregation routers and the plurality of backbone routers are configured to route both intra-data center data packets and Internet data packets.

[0215] D14. A method according to any of parts D2-D13, wherein the ingress aggregation router maintains a complete IP routing table.

[0216] D15. The method according to section D14, wherein the complete IP routing table includes both an Internet routing table and a data center routing table.

[0217] D16. A method according to any of parts D14 and D15, wherein the complete IP routing table is generated by the Border Gateway Protocol (BGP).

[0218] D17. The method of section D16, wherein the BGP on the ingress aggregation router receives routes announced by at least one of a BGP route reflector or a BGP client.

[0219] D18. A method according to section D17, wherein the BGP selects a set of one or more equal-cost BGP next hops for each prefix in the routing table based on the received route.

[0220] D19. The method according to section D18, wherein the set of one or more equal-cost BGP next hops is an egress aggregation router.

[0221] D20. A method according to section D18, wherein the set of one or more equal-cost BGP next hops are endpoints other than egress sites in a legacy portion of the WAN.

[0222] D21. The method of section D10, wherein the ingress aggregation router and the ingress backbone router reside at the same WAN site.

[0223] D22. The method of any of sections D2-D21, wherein the ingress backbone router to which the encapsulated data packet is forwarded is selected based on a weighted traffic-directed routing calculation by the ingress aggregation router.

[0224] D23. A method according to section D22, wherein the weighted traffic-oriented routing calculation is performed by a unified WAN agent running as a process on the ingress aggregation router.

[0225] D24. A method according to section D23, wherein the unified WAN agent uses an HTTPS server to communicate with the controller.

[0226] D25. A method according to part D24, wherein the controller calculates weighted business-oriented routing calculations for the unified WAN agent.

[0227] D26. The method of section D25, wherein the unified WAN agent programs the weighted service-oriented routing on the ingress aggregation router based on the weighted service-oriented routing calculation of the controller.

[0228] D27. A method according to section D26, wherein the controller excludes the ingress backbone router with the shortest path delay from the ingress aggregation router to the egress site, which shortest path delay exceeds the optimal delay threshold, and then uses the single item maximum flow from the ingress aggregation router to the egress site to calculate the weight.

[0229] D28. A method according to any part of parts D2-D27, wherein the ingress backbone router does not store a complete IP routing table.

[0230] D29. A method according to any of parts D2-D28, wherein selecting an optimized traffic engineering channel from two or more channels also includes an ingress backbone router performing traffic engineering optimization.

[0231] D30. A method according to D29, wherein the engineering optimization includes measuring the traffic matrix TM and the network diagram.

[0232] D31. A method according to section D30, wherein the service matrix is ​​a collection of service trunks and the bandwidth for each trunk.

[0233] D32. A method according to section D31, wherein the service trunk is an aggregated service flow from an ingress backbone router to a destination site for a service class.

[0234] D33. A method according to section D32, wherein the service category is one of voice, interactive, best effort, and scavenger.

[0235] D34. A method according to D30, wherein the network graph is a dynamic topology consisting of sites, nodes and links.

[0236] D35. A method according to part D34, wherein the nodes and links have one or more attributes.

[0237] D36. A method according to section D35, wherein the one or more attributes are one of an interface address, a device role, a link operating band, a bandwidth reserved for RSVP-TE, a link metric, information about whether a link or node should be avoided, and link reliability information.

[0238] D37. A method according to section D29, wherein the business engineering optimization includes a path calculation phase and an optimization phase.

[0239] D38. A method according to section D37, wherein in the path calculation phase, online calculation of paths on the dynamic topology is performed for all service trunks.

[0240] D39. A method according to section D38, wherein in the optimization phase, the priority fairness optimization solver assigns service trunks to paths.

[0241] D40. A method according to section D38, wherein each service trunk is optimized differently.

[0242] D41. A method according to section D39, wherein the priority fairness optimization solver also includes linking four solvers in different combinations based on business categories.

[0243] D42. The method according to section D41, wherein the four solvers include a max-min fairness solver, a minimize cost solver, a minimize maximum utilization solver, and a distinct path solver.

[0244] D43. A method according to any part of parts D29-D42, wherein the ingress backbone router also includes a unified WAN agent.

[0245] D44. The method according to section D43, wherein the unified WAN agent also includes programming two or more traffic engineering channels on the ingress backbone router based on traffic engineering optimization.

[0246] D45. A method according to any part of parts D2-D44, wherein selecting an optimized traffic engineering channel from two or more channels also includes an ingress backbone router using an egress site label to determine one or more channels between the ingress backbone router and an egress backbone router, wherein the egress backbone router is located at the egress site.

[0247] D46. The method according to section D45, wherein forwarding the encapsulated data packet along the selected optimized traffic engineering channel further comprises the ingress backbone router forwarding the encapsulated data packet to the egress backbone router using the selected optimized traffic engineering channel.

[0248] D47. A method according to section D46, wherein the selected optimized traffic engineering channel terminates at an egress backbone router.

[0249] D48. The method according to section D47, wherein the egress backbone router uses the second label to perform segment routing.

[0250] D49. The method of section D48, wherein the egress backbone router removes the second label from the encapsulated data packet.

[0251] D50. The method according to section D46, wherein the selected optimized traffic engineering channel terminates at an egress aggregation router.

[0252] D51. The method of section D50, wherein the egress aggregation router removes the second label from the encapsulated data packet.

[0253] One or more specific embodiments of the present disclosure are described herein. These described embodiments are examples of currently disclosed technology. In addition, in order to provide a brief description of these embodiments, all features of the actual embodiments may not be described in the specification. It should be understood that in the development of any such actual implementation, as in any engineering or design project, many embodiment-specific decisions will be made to achieve the specific goals of the developer, such as meeting system-related and business-related constraints, which can change from one embodiment to another. In addition, it should be understood that such development work may be complex and time-consuming, but it will still be a common task of design, production and manufacture for those of ordinary skill in the art who benefit from the present disclosure.

[0254] The articles "one", "an" and "the" are intended to mean that there are one or more elements in the elements in the foregoing description. The terms "comprising", "including" and "having" are intended to be inclusive, and mean that there may be additional elements in addition to the listed elements. In addition, it should be understood that the reference to "one embodiment" or "embodiment" of the present disclosure is not intended to be interpreted as excluding the existence of additional embodiments that also combine the features presented. For example, any element described with respect to the embodiments herein can be combined with any element of any other embodiment described herein. The numerals, percentages, ratios or other values ​​stated herein are intended to include the value, as well as other values ​​that are "about" or "approximately" the stated value, as will be understood by those of ordinary skill in the art covered by the embodiments of the present disclosure. Therefore, the stated value should be interpreted broadly enough to cover values ​​that are at least close enough to the stated value to perform the desired function or achieve the desired result. The stated value includes at least the expected changes in a suitable manufacturing or production process, and may include values ​​within 5%, 1%, 0.1% or 0.01% of the stated value.

[0255] Those with ordinary skill in the art will recognize that equivalent constructions do not depart from the essence and scope of the present disclosure, and that various changes, substitutions, and modifications may be made to the embodiments disclosed herein without departing from the essence and scope of the present disclosure. Equivalent constructions including functional "means plus function" clauses are intended to cover structures described herein that perform the functions presented, including both structural equivalents that operate in the same manner and equivalent structures that provide the same functions. It is the express intent of the applicant not to invoke means plus function or other functional claims for any claim, except those in which the words "means for..." appear together with the associated functions. Each addition, deletion, and modification to the embodiments that falls within the meaning and scope of the claims will be included in the claims.

[0256] As used herein, the terms "approximately," "about," and "substantially" indicate that a quantity is close to the stated quantity and still performs the desired function or achieves the desired result. For example, the terms "approximately," "about," and "substantially" may refer to quantities that are within less than 5%, within less than 1%, within less than 0.1%, and within less than 0.01% of the stated quantity. In addition, it should be understood that any directions or reference frames in the foregoing description are merely relative directions or movements. For example, any references to "up" and "down" or "above" or "below" merely describe the relative positions or movements of the associated elements.

[0257] The present disclosure may be embodied in other specific forms without departing from its essence or characteristics. The described embodiments will be considered illustrative rather than restrictive. Therefore, the scope of the present disclosure is indicated by the appended claims rather than by the preceding description. Changes that come within the meaning and scope of the equivalents of the claims will be included within their scope.

Claims

1. A method for routing data, comprising: obtaining an encapsulated data packet having a first label, wherein the first label is an egress site label, wherein the encapsulated data packet is associated with a destination; selecting an optimized traffic engineering channel from two or more channels; Replacing the first label with the optimized service engineering channel label; as well as The data packets are forwarded along the optimized traffic engineering channel.

2. The method of claim 1, wherein the encapsulated data packet is received by an ingress backbone router from an ingress aggregation router.

3. The method according to any one of claims 1 or 2, wherein encapsulating the data packet further comprises: A second tag is added to the data packet.

4. The method according to any one of claims 2 or 3, wherein the ingress backbone router to which the encapsulated data packet is forwarded is selected by the ingress aggregation router based on a weighted traffic-oriented routing calculation.

5. The method according to any one of claims 2 to 4, wherein selecting the optimized traffic engineering channel from the two or more channels further comprises: The ingress backbone router performs service engineering optimization.

6. The method according to any one of claims 2 to 5, wherein selecting the optimized traffic engineering channel from the two or more channels further comprises: The ingress backbone router uses the egress site label to determine one or more channels between the ingress backbone router and an egress backbone router, wherein the egress backbone router is located at an egress site.

7. The method of claim 6, wherein forwarding the encapsulated data packet along the optimized traffic engineering channel further comprises: The ingress backbone router forwards the encapsulated data packet to the egress backbone router using the optimized traffic engineering channel.

8. The method according to any one of claims 6 or 7, wherein the traffic engineering channel is terminated at the egress backbone router.

9. The method according to any one of claims 6 to 8, wherein the egress backbone router uses the second label to perform segment routing.

10. A unified wide area network (WAN), comprising: A backbone router, the backbone router comprising a traffic engineering module, wherein the traffic engineering module provides a traffic engineering channel between the backbone router and a destination router for the encapsulated data packets; as well as An aggregation router comprises a service steering module, wherein the service steering module encapsulates the encapsulated data packet and forwards the encapsulated data packet to the backbone router.

11. The unified WAN of claim 10, wherein the backbone router to which the encapsulated data packet is forwarded is selected by the traffic steering module based on a weighted traffic steering routing calculation.

12. The unified WAN according to any one of claims 10 or 11, wherein the business steering module encapsulates the data packet with the destination further comprising: A first tag is added to the data packet.

13. The unified WAN of claim 12, wherein the first label is an egress site label.

14. The unified WAN according to any one of claims 10 to 13, wherein the service steering module encapsulates the data packet with the destination further comprising: A second tag is added to the data packet.

15. The unified WAN of claim 14, wherein the second label is a node segment identifier (node ​​SID).

16. The unified WAN according to any one of claims 10 to 15, wherein the traffic engineering module selects an optimized traffic engineering channel from two or more channels by performing traffic engineering optimization.

17. The unified WAN of claim 16, wherein selecting the optimized traffic engineering channel from the two or more channels further comprises: The backbone router uses the egress site label to determine one or more traffic engineering channels for use between the backbone router and the destination router, wherein the destination router is located at the egress site.

18. The unified WAN of claim 17, wherein the backbone router replaces the egress site label with the optimized traffic engineering channel.

19. A method for routing data packets by an ingress backbone router, comprising: Retrieving an encapsulated data packet having a first label and a second label, wherein the first label is an egress site label and the second label is a node segment identifier (node ​​SID); Determine whether the business engineering channel to the egress site is available; When the traffic engineering channel to the egress site is available, replacing the first label with the traffic engineering channel and forwarding the encapsulated data packet along the traffic engineering channel; as well as When the traffic engineering channel to the egress site is unavailable, the first label is removed and the encapsulated data packet is forwarded using the node SID.

20. The method according to claim 19, wherein the ingress backbone router further comprises: A unified WAN agent, and wherein the unified WAN agent removes the first tag when the traffic engineering channel is unavailable, and wherein the unified WAN agent replaces the first tag with the traffic engineering channel when the traffic engineering channel is available.

Images