Automatic application-based multi-path routing for SD-WAN services

Abstract

Disclosed are automatic application-based multi-path routing for SD-WAN services. This application discloses example network devices, systems, and methods. In one example, a network device includes a memory configured to store information associated with one or more service level agreements (SLAs) for applications in a software-defined wide area network (SD-WAN), and an application-based multi-path routing (AMR) module including processing circuitry. The AMR module is configured to identify one or more applications for AMR based on criteria, where each of the criteria is associated with a corresponding attribute of the applications. The AMR module is configured to determine a violation of one of the SLAs on each WAN link associated with a first application of the identified one or more applications. The AMR module is configured to apply AMR to the first application in response to determining the violation.

Description

[0001] This application claims priority to U.S. Patent Application No. 17 / 473,561, filed September 13, 2021, the entirety of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates to computer networks. BACKGROUND

[0003] A computer network is a collection of interconnected computing devices that can exchange data and share resources. In a packet-based network such as the Internet, computing devices transmit data by dividing it into variable-length chunks called packets, which are individually routed over the network from a source device to a destination device. The destination device extracts the data from the packets and assembles it into its original form.

[0004] Network providers and enterprises can use software-defined networking (SD-WAN) in wide area networks to manage network connections between distributed locations, such as remote branch offices or central offices or data centers. SD-WAN extends SDN to enable enterprises to quickly and efficiently create connections over a WAN, which can include the Internet or other transport networks that provide various WAN connection types, such as Multiprotocol Label Switching (MPLS)-based connections, mobile network connections (e.g., 3G, Long-Term Evolution (LTE), 5G), Asymmetric Digital Subscriber Line (ADSL), and the like. Such connections are often referred to as “WAN links” or more simply as “links.” SD-WAN is considered a connectivity solution that leverages the above or other wide area network connection types, implemented with WAN links as an overlay on top of traditional WAN access.

[0005] SD-WAN services enable users, such as enterprises, to use WAN links to meet business and customer demands. In an SD-WAN environment, low-priority traffic can use less expensive Internet-based WAN links, while more important traffic can be transmitted over higher quality WAN links, such as those provided by MPLS networks. WAN link usage can also be allocated by application. With SD-WAN solutions, enterprise customers can mix and match cost optimization with service level agreement (SLA) requirements as they see fit. Users can expect their applications to experience connectivity with an acceptable level of quality, often referred to as quality of experience (QoE). QoE can be measured based on various performance metrics of a link, including latency, delay (interframe gap), jitter, packet loss, and / or throughput (e.g., bandwidth). Users can define desired levels of one or more metrics of QoE (e.g., SLAs) that the user expects in a service contract with a service provider. SLA metrics are typically values that the user can provision, derived through trial and error or benchmarking environments in comparison to user experience or actual best application metrics. SUMMARY

[0006] Generally, this disclosure describes techniques for automatically identifying which applications should have application-based multi-path routing (AMR) applied when delivering application traffic over an SD-WAN, and applying AMR to the identified applications when one or more conditions occur. According to the techniques of this disclosure, a network device can apply AMR to an identified application even if AMR is not pre-provisioned for the identified application.

[0007] For example, a network device can use criteria that can be associated with corresponding attributes of an application. In some examples, each of the criteria can have a corresponding weight. The criteria can be pre-determined by, for example, a network device manufacturer, or can be provisioned by a network operator, administrator, or customer of the SD-WAN service. In some examples, the criteria and / or the weights of the criteria can be dynamically and automatically adapted based on network conditions and / or the SD-WAN deployment, such that the criteria and / or the weights can change without operator input. The network device can identify which applications should be eligible for AMR based on the criteria. Then, when a service level agreement (SLA) of each WAN link associated with an application that is eligible for AMR is violated, the network device can apply AMR even if AMR is not pre-provisioned for the application. For example, the network device can apply AMR to the application by creating copies of application packets and sending the copies of the application packets over two or more WAN links.

[0008] It is a complex and tedious process to manually determine which applications should be provisioned for AMR and to provision such applications because thousands of applications can be providing traffic on the network. If AMR is pre-provisioned for some applications and a new application for which QoE is important is added to the network without AMR being pre-provisioned for the new application, then the QoE of the new application will be compromised if the SLA is not met on all WAN links associated with the new application. By automatically identifying which applications should be eligible candidates for AMR and applying AMR to the application even when AMR has not been pre-provisioned to the application, network devices can save network administrators countless hours of analyzing applications to determine which applications should be provisioned for AMR while also reducing the impact on QoE when SLAs are violated. In this way, example techniques provide technical solutions to technical problems related to determining which applications should be enabled for AMR and applying AMR to the applications even when AMR has not been pre-provisioned to the applications, and incorporate these techniques into practical applications to identify applications and apply AMR to the applications.

[0009] In one example, a network device includes a memory configured to store information associated with one or more service level agreements (SLAs) for applications in a software-defined wide area network (SD-WAN) and an application-based multipath routing (AMR) module including processing circuitry, the AMR module configured to: identify one or more applications for AMR based on criteria, wherein each of the criteria is associated with a corresponding attribute of the applications; determine a violation of one of the SLAs on each WAN link associated with a first application of the identified one or more applications; and apply AMR to the first application in response to determining the violation.

[0010] In one example, a method includes: identifying, by processing circuitry and based on criteria, one or more applications for application-based multipath routing (AMR) in a software-defined wide area network (SD-WAN), wherein each of the criteria is associated with a corresponding attribute of the applications; determining, by the processing circuitry, a violation of a service level agreement (SLA) for a first application of the identified one or more applications on each WAN link associated with the first application; and applying, by the processing circuitry and in response to determining the violation, AMR to the first application.

[0011] In one example, a non-transitory computer-readable storage medium comprising instructions that, when executed, cause a processing circuit to: identify one or more applications for application-based multi-path routing (AMR) in a software-defined wide-area network (SD-WAN) based on criteria, wherein each of the criteria is associated with a corresponding attribute of an application; determine a violation of a service level agreement (SLA) for a first application of the identified one or more applications on each WAN link associated with the first application; and responsive to determining the violation, apply AMR for the first application.

[0012] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a block diagram illustrating an example software-defined wide-area network (SD-WAN) system implemented in a network in accordance with the techniques of this disclosure.

[0014] Figure 2 is a block diagram illustrating an example SD-WAN edge device in more detail in accordance with the techniques described in this disclosure.

[0015] Figure 3 is a table diagram illustrating an example prioritization of applications.

[0016] Figure 4 is a table diagram illustrating an example of when to apply AMR.

[0017] Figure 5 is a flow diagram illustrating an example of automatically identifying applications for AMR and applying AMR techniques.

[0018] Figure 6 is a flow diagram illustrating another example of automatically identifying applications for AMR and applying AMR techniques.

[0019] Throughout the text and drawings, like reference numerals refer to like parts. DETAILED DESCRIPTION

[0020] Figure 1is a block diagram illustrating an example software-defined wide-area network (SD-WAN) system implemented in a network in accordance with the techniques of this disclosure. The SD-WAN system 100 includes transport networks 110A-110N (collectively, “transport networks 110”) for connecting sites attached to the transport networks and for transporting network traffic between such attached sites. One or more service providers can deploy the transport networks 110, and thus, the transport networks 110 can alternatively be referred to as “service provider networks.” Sites attached to the service provider networks can be referred to as “subscriber sites.” As used herein, the terms “subscriber,” “customer,” and “tenant” can be used interchangeably. The SD-WAN system 100 can be configured to implement the techniques disclosed herein to identify applications eligible for AMR and apply AMR to such applications.

[0021] A service provider uses the SD-WAN system 100 to provide an SD-WAN service 101 to its subscribers or organizations authorized by the subscribers, which can include, for example, cloud providers, cloud networks, and subscriber partners. The SD-WAN service 101 provides a virtual overlay network that enables application-aware, coordinated connectivity to deliver IP packets between sites associated with a subscriber in accordance with a policy. The service provider can provide multiple SD-WAN services.

[0022] The SD-WAN system 100 includes a service orchestrator 102, an SD-WAN controller 104, and a plurality of SD-WAN edges 108A-108C (hereinafter “SD-WAN edges” and collectively “SD-WAN edges 108”) that implement the SD-WAN service 101. The SD-WAN edges 108 are connected to each other by the transport networks 110. The control and ownership of the service orchestrator 102, the SD-WAN controller 104, the SD-WAN edges 108, and the transport networks 110 can be distributed among one or more service providers, subscribers, enterprises, or other organizations. However, an SD-WAN service provider uses all of these components to provide the SD-WAN service 101. The SD-WAN service provider can be an enterprise, a network / internet service provider, a cloud provider, or other entity.

[0023] In general, the service orchestrator 102 manages SD-WAN services. The service orchestrator 102 can control, fulfill, configure, monitor usage, guarantee, analyze, protect, modify, reconfigure policies, and apply policies to SD-WAN services. The service orchestrator 102 can establish application-based forwarding over the transport network 110 based on security policies, quality of service (QoS) policies, QoE policies, and / or business or intent-based policies. The service orchestrator 102 can include or represent a network service orchestrator (NSO). The service orchestrator 102 is aware of the resources of the SD-WAN system 100 and can implement, for example: tenant site and service management; end-to-end traffic coordination, visibility, and monitoring; physical network function (PNF) and / or virtual network function (VNF) management; policy and SLA management (PSLAM) to implement SD-WAN functionality; routing management for managing routing operations, including creating virtual private networks, enabling routing on the SD-WAN edges 108, and interfacing with route reflectors and routers; telemetry services that provide an interface used by the fault monitoring and performance monitoring systems to collect service check results from telemetry agents; and network activation functions to implement device provisioning. At least some of the above functions can be performed by components of the separate or integrated SD-WAN controller 104.

[0024] The SD-WAN controller 104 can include or represent a network service controller (NSC). In general, the service orchestrator 102 interacts with the SD-WAN controller 104 to manage the SD-WAN edges 108 to create and operate end-to-end SD-WAN managed services between the SD-WAN edges 108 over the transport network 110. The SD-WAN controller 104 can provide topology and SD-WAN edge 108 lifecycle management functions. For example, the SD-WAN controller 104 provides PNF / VNF management for the SD-WAN edges 108 managed by the service orchestrator 102. For example, the SD-WAN controller 104 can configure network configurations of the SD-WAN edges 108, configure policies on the SD-WAN edges 108, and so on. The SD-WAN controller 104 can monitor status and performance data of the SD-WAN edges 108 and the WAN links 142A-A to 142N-N (collectively, “WAN links 142”) and provide this information to the service orchestrator 102. In other words, the SD-WAN controller 104 can communicate with the SD-WAN edges 108 to determine operational status of the WAN links 142 over the transport network 110 and obtain QoS / QoE performance metrics for the WAN links 142. As described in further detail, the SD-WAN system 100 can modify traffic patterns based on the performance metrics of the WAN links to better meet SLA requirements of the SD-WAN services in the SD-WAN system 100.

[0025] In various examples of the SD-WAN system 100, the service orchestrator 102 and the SD-WAN controller 104 can be combined, for example, to form a single service orchestration platform with separate service orchestration and domain orchestration layers, deployed as separate devices or appliances, or each service orchestration platform can be distributed among one or more components executing on one or more servers deployed in one or more locations. The service orchestrator 102 can be a scalable and cloud-deployable platform. For example, a service provider of the SD-WAN services in the SD-WAN system 100 can deploy the service orchestrator 102 to a provider site or a public cloud, a private cloud, or a hybrid cloud. As such, the operations and functions attributed to the service orchestrator 102 in this disclosure can be performed by the separate SD-WAN controller 104, and vice versa. In some example architectures, aspects of service orchestration and SD-WAN control can also be distributed from the service orchestrator 102 and the SD-WAN controller 104, respectively, in the SD-WAN edges 108.

[0026] Administrators and applications can interact with the service orchestrator 102 using a northbound interface, such as a RESTful interface (web-based REST API), a command-line interface, a portal or graphical user interface, a web-based user interface, or other interface of the service orchestrator 102 (not shown in Figure 1 The service orchestrator 102 can communicate with the SD-WAN controller 104 via a southbound interface, which can be a northbound interface of the SD-WAN controller, such as a RESTful interface, a command-line interface, a graphical user interface, or other interface of the service orchestrator 102 (not shown in Figure 1

[0027] The network links 140 connect the SD-WAN edges 108 to the transport networks 110. The network links 140 and the transport networks 110 constitute the underlay network of the SD-WAN service 101 and provide underlay connections between pairs of SD-WAN edges 108. For example, the transport network 110A and the transport network 110N provide separate underlay connections (not shown in Figure 1 The underlay connections can be public or private and can be network services, such as label switched paths (LSPs), Ethernet services, and IP services, public Internet services, or other services that implement overlay WAN links. The cost of using the underlay connections can be a flat rate or usage-based. Each underlay connection can have a bandwidth limit, performance metrics (e.g., latency, loss, jitter, etc.). The SD-WAN service 101 can be deployed using underlay connections based on multiple different types of network services. For example, the SD-WAN service 101 can be deployed using underlay connections based on LSPs, Ethernet services, and IP services. Figure 1 ​In the example of FIG. 1, the underlay connection from SD-WAN edge 108A to SD-WAN edge 108C via transport network 110A can be an LSP for IP-VPN, while the underlay connection from SD-WAN edge 108A to SD-WAN edge 108C via transport network 110N can be an IPSec tunnel over the Internet. This diversity can be advantageous for the SD-WAN service by facilitating redundancy and by providing differentiated service capabilities to match cost / performance to application needs / SLAs for different traffic using the SD-WAN service. For example, SD-WAN edge 108A can direct low-cost traffic via the Internet, while directing traffic for applications that require low latency (e.g., IP telephony) over an LSP. The underlay connections can be created and / or managed by the SD-WAN service provider or the SD-WAN service 101 subscriber, with the service orchestrator 102 being informed of the underlay connections. The service orchestrator 102 obtains link data for WAN links 142, including bandwidth limitations (if any) for WAN links 142. The service orchestrator 102 can obtain the link data from the SD-WAN controller 104, receive configuration data with the link data, or obtain the link data from another network controller or from the SD-WAN edges 108. WAN links 142 are depicted and shown as bidirectional, but each of WAN links 142 can represent two separate WAN links, one in each direction.

[0028] The SD-WAN system 100 shows multiple sites associated with a subscriber of the SD-WAN service 101 provider and attached to subscriber-facing interfaces of the SD-WAN edges 108. These sites can be referred to as subscriber sites, and they make up a subscriber network because the SD-WAN service 101 interconnects the multiple sites to form a single network. Figure 1 The SD-WAN system 100 in the example of FIG. 1 includes sites 106A-106B and can optionally include any of sites 106C, hub 112, cloud 114, or cloud services 116. In some cases, the “subscriber” and the SD-WAN provider are the same entity, as is the case for enterprise deployments and management of the SD-WAN system 100.

[0029] Each of sites 106A through 106C refers to a subscriber location and can represent, for example, a branch office, a private cloud, an on-premises branch, an enterprise hub, or a cloud branch. Provider hub 112 represents a multi-tenant hub device located at a point of presence (PoP) on the service provider network. Provider hub 112 can terminate overlay tunnels used for overlay networks, which can be of various types, such as MPLS-based general routing encapsulation (MPLSoGRE), IPSec-based MPLSoGRE (MPLSoGREoIPsec), and MPLS-based User Datagram Protocol (MPLSoUDP) tunnels. Provider hub 112 can be a hub in a hub-and-spoke architecture used in some example deployments of SD-WAN service 101.

[0030] Cloud 114 refers to public, private, or hybrid cloud infrastructure. Cloud 114 can be a virtual private cloud within a public cloud. Cloud service 116 is a resource or higher-level service provided to subscribers by a cloud service provider via SD-WAN service 101. Cloud service 116 can be, for example, Software as a Service (SaaS), Platform as a Service (PaaS), Infrastructure as a Service (IaaS), Storage as a Service, or other types of cloud services. Cloud service 116 can be provided by the infrastructure of cloud 114.

[0031] Internet 118 refers to web and / or internet connectivity services provided via the web. In this example, the SD-WAN edge 108B includes an internet breakout 120, and application flows are distributed to the internet breakout 120 according to policies.

[0032] Each of the SD-WAN edges 108 includes physical network functions or virtual network functions for implementing the SD-WAN service 101. In various examples, each of the SD-WAN edges 108 may be one or more VNFs or PNFs, for example, located within any of a service provider data center, provider hub, client, or cloud provider endpoint. Each of the SD-WAN edges 108 may be a router, a security device such as a firewall, a gateway, a WAN acceleration device, a switch, a cloud router, a virtual gateway, a cloud virtual gateway, an SD-WAN device, or other devices that implement aspects of the SD-WAN service 101.

[0033] In various examples, each of the SD-WAN edges 108 can be an on-premise branch, which is a PNF placed at a user branch site in a hub-and-spoke topology or a full mesh topology; a cloud branch, which is a VNF in a virtual private cloud (VPC) (or equivalent term) of a subscriber located in a public cloud; a PNF or VNF located in a service provider cloud operating as a hub device to establish tunnels with branch sites (the hub device is multi-tenant, i.e., shared between multiple sites by using virtual routing and forwarding instances configured on multiple sites); a PNF or VNF located in an enterprise operating as an enterprise hub providing additional hub-like functionality for ordinary branch sites (e.g., an anchor point for branches created as dynamic virtual private networks (VPNs), providing central egress options within the enterprise, hosting data center departments, importing routing protocol routes to create dynamic LAN segments, and meshing with other enterprise hubs belonging to the same tenant / subscriber). Each of the SD-WAN edges can be located at any of the sites 106, hubs 112, clouds 114, or cloud services 116.

[0034] The SD-WAN edges 108 are logically located at the boundary between the provider SD-WAN service 101 and the subscriber network. The SD-WAN edges 108 have network-side interfaces for the underlying connections and subscriber-side interfaces for communicating with the subscriber network. As described above, the SD-WAN edges 108 can have multiple paths (different underlying connections) to each other. For example, in a hub-and-spoke deployment, the SD-WAN edge 108A has multiple paths, each via a different one of the transport networks 110, to the SD-WAN edge 108C of the hub 112. The interfaces of the SD-WAN edges 108 can be primarily for the underlying connections of user data traffic, but the interfaces can also be used for management (operations, maintenance, and administration (OAM)) traffic, e.g., to send performance metrics to the service orchestrator 102 and to receive policies, device configurations, and other configuration data from the service orchestrator 102.

[0035] The service orchestrator 102 can provide and establish overlay tunnels between the SD-WAN edges 108 to implement the SD-WAN service 101 topology. In Figure 1 In examples, any of the WAN links 142 can be implemented in part using point-to-point overlay tunnels, e.g., for virtual private networks. The overlay tunnels inherit the performance characteristics of the underlying connections. The overlay tunnels can be encrypted or unencrypted. The SD-WAN edges 108 can implement the overlay tunnels using any of a variety of encapsulation types, e.g., MPLS, MPLSoGRE, IP-in-IP, MPLSoUDP, MPLSoGREoIPSec, IPSec, GRE.

[0036] The SD-WAN edge 108 uses WAN links 142 to send application traffic to other SD-WAN edges 108 over the SD-WAN service 101. The WAN links 142 typically, but not necessarily, traverse different underlay connections between the SD-WAN edges 108. N WAN links 142A-A to 142A-N connect the SD-WAN edge 108A and the SD-WAN edge 108C. In Figure 1 In the example, each of the WAN links 142A-A to 142A-N traverse a different one of the transport networks 110. Similarly, N WAN links 142N-A to 142N-N connect the SD-WAN edge 108B and the SD-WAN edge 108C, each via a different one of the transport networks 110. In a full mesh topology (not shown), additional WAN links would connect the SD-WAN edges 108A, 108B. The WAN links 142 can also be referred to as "overlay connections," "virtual connections," "tunnel virtual connections," "SD-WAN links," or other terminology describing WAN links used to implement an SD-WAN service.

[0037] According to techniques in accordance with some aspects of the application, the SD-WAN edge 108B can identify one or more applications for AMR in a software-defined wide area network (SD-WAN) based on criteria, where each of the criteria is associated with a corresponding attribute of the applications. In some examples, the criteria have weights. The SD-WAN edge 108B can determine a violation of one of the SLAs on each WAN link associated with a first application of the one or more applications. The SD-WAN edge 108B can apply the AMR to the first application in response to determining the violation.

[0038] In some examples, the SD-WAN edge 108 can be configured to identify applications for AMR, e.g., applications associated with application traffic forwarded through the SD-WAN edge 108. For example, the SD-WAN edge 108B can be configured to identify QoE relatively important applications (also referred to herein as relatively important applications). In some examples, the SD-WAN edge 108B can be configured to store criteria, each of the criteria associated with a corresponding attribute of an application, and in some examples, each criterion has a corresponding weight. The SD-WAN edge 108B can identify one or more applications for AMR based on the criteria. For example, the SD-WAN edge 108B can compare attributes of an application to the criteria, and if one or more attributes of the application match any of the criteria, the SD-WAN edge 108B can identify the application as a candidate for AMR or as eligible for AMR. If none of the attributes of the application match any of the criteria, the SD-WAN edge 108B can identify the application as not a candidate for AMR or not eligible for AMR. In this way, the SD-WAN edge 108B can use the criteria to automatically determine whether an application is a candidate for AMR.

[0039] The SD-WAN edge 108B can monitor each WAN link associated with an application that has been identified for AMR and determine a violation of an SLA on each WAN link associated with a first application that has been identified as eligible for AMR. The SD-WAN edge 108B can apply AMR to the first application in response to determining the violation. The SD-WAN edge 108B can do so even if the first application was not pre-provisioned with AMR. That is, the first application was not explicitly predefined as an application and implementing AMR in configuration data of the SD-WAN edge 108B.

[0040] When AMR is applied to the first application, the SD-WAN edge 108B can replicate application data packets of the first application on the SD-WAN edge 108B and can forward a copy of each application data packet on each of the two or more WAN links. For example, the SD-WAN edge 108B can replicate application data packets of the first application and forward copies of the application data packets to the SD-WAN edge 108C via both WAN links 142N-A and 142N-N. In this way, the SD-WAN edge 108C can receive multiple copies of the same application data packet. When the SD-WAN edge 108C receives multiple copies of the same application data packet from the SD-WAN edge 108B, the SD-WAN edge 108C can keep the first arriving application data packet and delete or ignore the second arriving application data packet. In this way, the QoE of the application can not be impacted as much as if the application traffic were carried using only a single WAN link.

[0041] In some examples, the SD-WAN system 100 can not have sufficient resources to apply AMR to applications identified as eligible for AMR. Accordingly, in some examples, the SD-WAN edge 108B can be configured to determine whether available resources are sufficient to support AMR for a first application and to apply AMR based further on the available resources being sufficient to support AMR for the first application. In some examples, each of the multiple applications that have been identified for AMR can experience a SLA violation for each WAN link associated with the application. As such, in some examples, the SD-WAN edge 108B can be configured to prioritize the applications identified for AMR. For example, as described above, a weight can be assigned to each of the criteria in the standard that can be used to determine a priority for AMR among the applications identified as eligible for AMR. The SD-WAN edge 108B can determine a priority for each of the applications identified for AMR based on a cumulative weight of each criterion with a corresponding attribute satisfied by the application. In this manner, if there are sufficient resources to apply AMR to one or more of the identified applications, but not sufficient resources to apply AMR to all of the identified applications, the SD-WAN edge 108B can use the priority of each of the identified applications to determine which of the identified applications should have AMR applied. For example, the SD-WAN edge 108B can apply AMR to higher priority applications and refrain from applying AMR to lower priority applications. As another example, the SD-WAN edge 108B can apply AMR to additional applications in priority order until resource usage meets a threshold. In some examples, the SD-WAN edge 108B can reserve a portion of the available resources or network resources for other purposes, such as new applications. In this example, the SD-WAN edge 108B can refrain from applying AMR to one or more lower priority applications even though there are sufficient resources to apply AMR to the one or more lower priority applications.

[0042] The service orchestrator 102 can use the SD-WAN controller 104 to deploy the SD-WAN service 101 in various architectural topologies including mesh and hub-and-spoke. A mesh topology is one in which traffic can flow directly from any site 106 to another other site 106. In a dynamic mesh, the SD-WAN edge 108 holds resources for implementing a full mesh topology. All sites in a full mesh network are included in the topology, but site-to-site VPNs are only enabled when traffic exceeds a user-defined threshold, referred to as a dynamic VPN threshold. Sites in a mesh topology can include sites 106, clouds 114, and / or cloud services 116. In a hub-and-spoke topology, all traffic passes through the hub 112, more specifically, through the SD-WAN edge 108C deployed at the provider hub 112. By default, traffic to the Internet also flows through the provider hub 112. In a hub-and-spoke topology, network services (e.g., firewall or other security services) can be applied at the central hub 112 location, which allows all network traffic of the SD-WAN service 101 to be processed using a single site’s network services. The SD-WAN service 101 can have a regional hub topology that combines full mesh and hub-and-spoke, which uses one or more regional hubs to connect multiple branches to a wider mesh.

[0043] In some examples, the SD-WAN controller 104 includes a route reflector (not shown) to facilitate routing in the SD-WAN service 101. The route reflector forms overlay Border Gateway Protocol (BGP) sessions with the SD-WAN edges 108 to receive, insert, and reflect routes.

[0044] The SD-WAN edge 108 receives incoming network traffic from a corresponding subscriber site and applies the SD-WAN service 101 to forward the network traffic to another one of the SD-WAN edges 108 via one of the WAN links 142. The SD-WAN edge 108 receives network traffic on a WAN link 142 and applies the SD-WAN service 101 to forward the network (e.g., to another one of the SD-WAN edges 108 where the SD-WAN edge is a hub) or to a destination subscriber site via one of the WAN links 142.

[0045] To apply the SD-WAN service 101, the SD-WAN edge 108 processes network traffic according to routing information, policy information, performance data, and service characteristics of the WAN links 142, which can be derived at least in part from the performance, bandwidth constraints, and behavior of the underlying connections. For example, the SD-WAN edge 108 uses dynamic path selection to direct network traffic to different WAN links 142 to attempt to meet QoS / QoE requirements defined in SLAs and configured for the SD-WAN service 101 in the SD-WAN edge 108, or to route around a failed WAN link. For example, the SD-WAN edge 108A can select WAN link 142A-A, which is a low-latency MPLS path (in this example), for VoIP traffic, and select WAN link 142A-N, which is a low-cost, broadband Internet connection, for file transfer / storage traffic. The SD-WAN edge 108 can also apply traffic shaping. The terms "link selection" and "path selection" refer to the same operation of selecting a WAN link for an application, and are used interchangeably.

[0046] The SD-WAN edges 108 process and forward received network traffic for the SD-WAN services 101 according to the policy and configuration data from the service orchestrator 102, routing information, and current network conditions including underlying connection performance characteristics. In some examples, the service orchestrator 102 can push SLA parameters, path selection parameters, and related configuration to the SD-WAN edges 108, and the SD-WAN edges 108 monitor the links for SLA violations and can apply AMR to applications identified for AMR, or switch the applications to a different one of the WAN links 142. The SD-WAN edges 108 can thereby implement data plane functionality for the SD-WAN services 101 on the underlying connections, in this example including applying AMR for QoE of the applications and switching the applications to different WAN links 142. If one of the SD-WAN edges 108 detects an SLA violation, the SD-WAN edge can or can not apply AMR as described throughout this disclosure, but can still report and send log messages to the service orchestrator 102 describing the SLA violation and the selected WAN link. The SD-WAN edges 108 can also aggregate, optionally average, and report SLA metrics for the WAN links 142 in log messages to the service orchestrator 102. In some examples, the service orchestrator 102 can receive the SLA metrics from the SD-WAN edges 108, determine that the SLA for an application has been violated, and perform path selection to select a new one of the WAN links 142 for the application that violates the SLA. The service orchestrator 102 can then configure one or more of the SD-WAN edges 108 to switch the application traffic for the application on the new WAN link. The SLA metric analysis, SLA evaluation, path selection, and link switching functionality are all performed by the SD-WAN system 100, but different examples of the SD-WAN system 100 can have different distributions of the control plane functionality between the service orchestrator 102 and the SD-WAN edges 108 than those just described. However, the functionality is described below primarily with respect to the SD-WAN edges 108.

[0047] The SD-WAN edges 108 can forward traffic on a per-application flow basis. Application flow packets can be identified using packet characteristics, such as layer 3 and layer 4 (e.g., TCP, UDP) header fields (e.g., source / destination layer 3 address, source / destination port, protocol), by deep packet inspection (DPI), or other flow identification techniques used to map packets to an application (or more specifically, an application flow). An application flow can include packets for multiple different applications or application sessions, and a single application can be split between multiple application flows (e.g., separate video and audio flows for a video conferencing application).

[0048] The SLA can specify the applicable application flow and can include policies for application flow forwarding. The SD-WAN edge 108 can identify the application flow and apply the appropriate policies to determine how to forward the application flow. For example, the SD-WAN edge 108 can use application-specific QoE and advanced policy-based routing (APBR) to identify the application flow and specify a path for the application flow by associating the SLA profile with a routing instance on which to send the application flow. The routing instance can be a virtual routing and forwarding instance (VRF) that is configured with interfaces for the WAN links 142.

[0049] The QoE is intended to improve user experience at the application layer by monitoring service level parameters and SLA compliance of application traffic, and facilitating placement of application data on a SLA-compliant WAN link 142 (or the highest SLA-compliant WAN link available), or facilitating use of AMR for applications identified for AMR. For example, the SD-WAN edge 108 and the service orchestrator 102 can monitor application traffic for SLA compliance for an application. In some examples, the SD-WAN edge 108 (independently or according to an indication from the service orchestrator 102) can move application traffic from a WAN 142 link that cannot meet SLA requirements to one of the WAN links 142 that meets the SLA requirements, or apply AMR for an application identified for AMR when all WAN links associated with the application cannot meet the SLA requirements.

[0050] To monitor SLA compliance of the links over which application traffic is sent, the service orchestrator 102 can cause the SD-WAN edge 108 to send inline probes along the WAN links 142 (in some cases, along with application traffic that has already been sent). These inline probes can be referred to as "passive probes." To identify the best available one of the WAN links 142 for an application in the event that the active WAN link fails to meet SLA criteria, the service orchestrator 102 monitors and collects SLA compliance data for other available WAN links 142 for the SD-WAN service 101. Probes sent by the service orchestrator 102 through other WAN links 142 to check SLA compliance can be referred to as "active probes." Active probes are executed based on probe parameters that are provided in some cases by a subscriber. Both active and passive probes measure end-to-end analytics for WAN links 142. Data collected through active and passive probing is used to monitor the network for sources of failure or congestion. If a violation is detected for any application or group of applications ("application group"), the service orchestrator 102 evaluates the composite probe metrics to determine a desirable, and in some cases, the best WAN link 142 that meets the SLA. As used herein, a reference to an application can refer to a single application or any application group.

[0051] Configuring the service orchestrator 102 to cause the SD-WAN system 100 to apply QoE for the SD-WAN service 101 can involve configuring multiple configuration files of various configuration file types to enable a user to parameterize QoE for various applications / application groups with traffic transported by the SD-WAN service 101. Configuration files generally include human-readable text that defines one or more parameters for a function, or associates a configuration file with other configuration files to parameterize a higher-level function. In various examples, the service orchestrator 102 can provide various configuration schemes for QoE parameterization for the SD-WAN service 101.

[0052] A subscriber or network administrator can interact with the service orchestrator 102 to create an SLA profile for an application, referred to herein as an "application SLA profile" or simply an "SLA profile." The SLA profile can include SLA configuration data, such as a traffic type profile, an indication of whether local breakout is enabled, an indication of a path preference (e.g., a preferred WAN link or WAN link type (e.g., MPLS, Internet, etc.) for the WAN link 142), an indication of whether failover is allowed when the active WAN link has an SLA violation of the SLA profile, criteria for failover (e.g., a violation of any SLA parameter or a violation of all SLA parameters required to trigger failover). In some examples, a network administrator can interact with the service orchestrator and / or the SD-WAN edge 108 to provision an AMR for selected applications, or to customize the criteria and / or weights used by the SD-WAN edge 108 to identify and prioritize applications for the AMR.

[0053] The SLA parameters can be included in an SLA metric profile, which is associated with or part of the SLA profile. The service orchestrator 102 and the SD-WAN edge 108 can use the SLA parameters to evaluate the SLA of the WAN link 142. The SLA parameters can include parameters such as throughput, latency, jitter, jitter type, packet loss, round trip delay, or other performance metrics of the traffic (which are related to and correspond to the performance metrics of the WAN link carrying such traffic). Throughput can refer to the amount of data sent or received by a site upstream or downstream in a period of time. Latency refers to the time it takes for a packet to travel from one designated point to another. Packet loss can be specified as a percentage of packets dropped by the network to manage congestion. Jitter is the difference between the maximum and minimum round trip times of packets.

[0054] The SLA configuration file can also specify SLA sampling parameters and rate limit parameters. The sampling parameters can include a session sampling percentage, an SLA violation count, a sampling period, and a switch cool down period. The session sampling percentage can be used to specify a matching percentage of sessions for which the service orchestrator should run passive probes. The SLA violation count is used to specify a number of SLA violations after which the service orchestrator should switch to a different one of the WAN links 142, or consider the SLA to be violated in determining whether to apply AMR for a given application. The sampling period can be used to specify a sampling period for counting SLA violations. The switch cool down period can be used to specify a waiting period after which a WAN link switch should occur if the active link comes back online after a failure, or after which AMR should be stopped for a given application. This parameter helps to prevent frequent switching of traffic between active and backup WAN links 142, or between AMR mode and normal mode.

[0055] The rate limit parameters can include a maximum uplink rate, a maximum uplink burst size, a maximum downlink rate, a maximum downlink burst size, and a loss priority. The maximum uplink rate can be used to specify a maximum uplink rate for all applications associated with the SLA configuration file. The maximum uplink burst size can be used to specify a maximum uplink burst size for all applications associated with the SLA configuration file. The maximum downlink rate can be used to specify a maximum downlink rate for all applications associated with the SLA configuration file. The maximum downlink burst size can be used to specify a maximum downlink burst size for all applications associated with the SLA configuration file. The loss priority can be used to select a loss priority based on which packets can be dropped or preserved when network congestion occurs. The probability of a packet being dropped by the network is higher or lower based on the loss priority value.

[0056] An SLA rule can be used to specify an application SLA profile, which includes all information needed to measure the SLA and identify whether any SLA violations have occurred. The SLA rule can contain a complete probe profile, a time period for which the application profile is to be applied, a preferred SLA profile, and other SLA parameters described above (e.g., SLA sample parameters, rate limit parameters, metric profile). The SLA rule is associated with an application or application group and will be its SLA profile. In other words, the SLA profile for an application can be a specific SLA rule (e.g., “SLA3”) as configured in the service orchestrator 102. In some cases, the SLA rule can be associated in this manner by being associated with an APBR rule that matches the identified application or application group. As described above, in some examples, the service orchestrator 102 can push SLA parameters, path selection parameters, routing information, routing and interface data, and related configurations to the SD-WAN edge 108, and the SD-WAN edge 108 monitors the links for SLA violations and can apply AMRs to the applications identified for AMR or switch the applications to a different one of the WAN links 142.

[0057] An SLA violation occurs when the performance of a link falls below the acceptable level specified by the SLA. To attempt to meet the SLA, the SD-WAN system 100 can monitor the network for sources of failure or congestion. If the SD-WAN system 100 determines that an SLA violation has occurred, the SD-WAN system 100 can determine a replacement path to select the best WAN link 142 that meets the SLA. Similarly, the SD-WAN edge 108 can monitor for SLA violations. If one of the SD-WAN edges 108 determines that an SLA violation or breach has occurred on all WAN links associated with an application identified for AMR, the SD-WAN edge can apply an AMR to the application.

[0058] A covered path includes a WAN link 142 used to transmit application traffic for an application. The SD-WAN system 100 can assign an application to a particular WAN link 142 based on SLA metrics for the WAN link 142. A destination group is a group of multiple covered paths that terminate at a destination.

[0059] Generally, the service orchestrator 102 configures the SD-WAN edge 108 to recognize application traffic for an application and the service orchestrator 102 specifies a path for the particular traffic by associating an SLA profile with a routing instance through which the SD-WAN edge 108 transmits application traffic to meet the rules of an APBR profile.

[0060] The APBR is implemented by the service orchestrator 102 managing the SD-WAN edge 108. The APBR configuration file specifies matching traffic types, e.g., by listing one or more applications or application groups. The APBR configuration file can include multiple APBR rules, each of which specifies one or more applications or application groups. A rule is considered to match if network traffic matches the specified application. SLA rules can be associated with the APBR rules to specify how the matching traffic should be handled for QoE. The APBR rules can also specify a routing instance for the SD-WAN edge 108 to use to route traffic matching the APBR rules. The routing instance can have interfaces for one or more WAN links 142. The service orchestrator 102 configures the SD-WAN edge 108 with the APBR configuration file (or configuration data derived therefrom) to cause the SD-WAN edge 108 to implement the SD-WAN service 101 using APBR according to the APBR configuration file.

[0061] In some examples, the SD-WAN edge 108 (e.g., the SD-WAN edge 108A) processes a data packet received on an interface to identify an application for the data packet. The SD-WAN edge 108A can apply the APBR configuration file to attempt to match the application to an APBR rule therein. If no matching APBR rule is found, the SD-WAN edge 108A normally forwards the data packet. However, if a matching APBR rule is found, the SD-WAN edge 108A uses the routing instance specified in the APBR rule to route the data packet.

[0062] The routing instance has interfaces associated with one or more links that the routing instance uses to send and receive data. The routing instances configured in the SD-WAN edge 108 and that can be associated with APBR rules have interfaces for WAN links 142 to send and receive application traffic. These interfaces can be the interfaces for the underlying connections.

[0063] The SD-WAN edge 108 can route traffic using different links based on link preferences determined using the SLA rules 122. In some cases, the service orchestrator 102 determines application performance on WAN links in the WAN links 142 by calculating a score based on latency, round trip time, jitter, packet loss, and / or other factors. Based on the respective scores of one or more of the WAN links 142, if the performance of the current link is below an acceptable level specified by one of the SLA rules 122, the service orchestrator 102 and the SD-WAN edge 108 can transfer application traffic to an alternative WAN link of the SD-WAN service 101. In some cases, the new WAN link is the WAN link determined to best meet the SLA requirements according to the scores. As already mentioned, the service orchestrator 102 can use probes to measure and monitor application performance on the WAN links 142.

[0064] In some examples, multiple WAN links 142 can meet the SLA requirements of an application. The SD-WAN system 100 can select a WAN link from among the multiple WAN links 142 that matches a user-configured link preference. The preference can be based at least in part on the link type and link priority of the WAN links 142. For example, for the SD-WAN edge 108A, the SD-WAN system 100 can select one of the WAN links 142A-A to 142A-N to the SD-WAN edge 108C that matches a preferred link type (e.g., MPLS). If there are multiple such WAN links 142 with the preference, the WAN link with the highest priority is selected. If no priority or link type preference is configured, a random path or a default path is selected. If no WAN link 142 meeting the SLA requirements is available, in the case where strict affinity is configured, the best available WAN link according to the highest SLA score and link type preference is selected. If multiple WAN links 142 meeting the SLA requirements are available, the one with the highest priority is selected. One or more of the WAN links 142 can be configured with a priority, which can be expressed in the configuration as an integer value representing the priority. The service orchestrator 102 prefers WAN links 142 with higher priority over WAN links 142 with lower priority.

[0065] In the service orchestrator 102, a user can configure a link type (e.g., IP or MPLS) and set a priority for the WAN links 142 for an application. For example, a user can define an APBR profile with WAN links 142 and configure the WAN links 142 with a link type / priority.

[0066] By associating APBR rules for a specified application or application group with an APBR profile, the service orchestrator 102 and the SD-WAN edge 108 enforce link preferences at the application or application group level to implement the SD-WAN service 101. The user can also specify link type preferences in the SLA rules and, in some cases, link type affinities. The SLA rules are attached to the APBR rules in order to associate the preferences with the applications specified in the APBR rules.

[0067] Based on the APBR profile, the SD-WAN edge 108 matches network traffic to the applications and application groups specified in the associated APBR rules and can, for example, forward the traffic to the next-hop addresses specified in the routing instances of the static routes and APBR rules that are also associated with the APBR profile. The SD-WAN system 100 can assign application traffic to a particular path / link based on the configured link types and preferences of the WAN links 142, and in some cases, the specified link type affinities used in the SLA rules (as described above).

[0068] For a preferred link type, the link type affinity can be strict or loose (optionally the default setting). For strict affinity, the SD-WAN system 100 selects a WAN link that is always of the preferred link type. For loose affinity, if no WAN link 142 exists that satisfies the SLA and is of the preferred link type, the service orchestrator 102 selects a link that does not have the preferred link type but otherwise satisfies the SLA.

[0069] The service orchestrator 102 can implement SD-WAN policy intents for the SD-WAN service 101 to facilitate better WAN link 142 utilization and efficient distribution of application traffic. The subscriber can set high-level SD-WAN policies that include one or more SD-WAN policy intents. Each SD-WAN policy intent can have the following parameters: source, destination, and SLA profile. The source is one or more source endpoints selected from a list of sites, site groups, departments, or a combination thereof. The SD-WAN policy intent applies to the selected source endpoints. The destination is a destination endpoint selected from a list of applications and predefined or custom application groups. The SD-WAN policy intent applies to the selected destination endpoints. The applications can be defined using, for example, network information (e.g., source or destination prefixes), by protocol, or by application name. The SLA profile can be defined as described above with the SLA parameters to be applied to the policy intent for which the SLA profile is set.

[0070] An SLA rule in SLA rules 122 specifies one or more applications. As used herein, this or other associations between an application and SLA parameters for the application means that the application has an SLA (or SLA rule). If an SLA parameter is violated, the SLA / SLA rule for the application is violated.

[0071] SD-WAN system 100 can determine the available bandwidth of one of WAN links 142 (e.g., WAN link 142A-A) in a number of ways. For example, SD-WAN system 100 can obtain link data indicating the available bandwidth or total bandwidth of WAN link 142A-A. SD-WAN controller 104 can provide link data to service controller 102, which can be obtained in part from SD-WAN edge 108. The link data can be configuration data for the underlying connection of WAN link 142A-A. SD-WAN system 100 can sum the bandwidth required by applications placed on WAN link 142A-A and calculate the difference between the total bandwidth of WAN link 142A-A and the sum of the bandwidth required by applications placed on WAN link 142A-A as the available bandwidth.

[0072] By identifying applications for AMR based on criteria, SD-WAN edge 108B can enable a network administrator to not have to determine the applications for which to provision AMR and not have to manually provision the applications. This can reduce the potential for human error in the process and enable AMR to be implemented more quickly, thereby providing better QoE, particularly for relatively more important applications or those that meet a particular application profile prioritized by the administrator for automatic AMR. Application profiles can be defined by the administrator using criteria and, in some cases, by criteria in combination with weights for the criteria.

[0073] Although primarily described as being performed by SD-WAN edge 108B, the techniques described herein for automatically identifying applications that are eligible for AMR can be performed by other SD-WAN edges 108 and, in part, by SD-WAN controller 104. For example, SD-WAN controller 104 can receive telemetry data indicating conditions of WAN links 142 and attributes of applications being processed by SD-WAN services 101, determine that an application is eligible for AMR, and direct one or more of SD-WAN edges 108 to apply AMR.

[0074] Figure 2 is a block diagram illustrating an example SD-WAN edge device in more detail in accordance with the techniques described in this disclosure. SD-WAN edge device 308 (“SD-WAN edge 308”) can represent Figure 1Any of the SD-WAN edges 108. The SD-WAN edge 308 is a computing device and can represent a PNF or a VNF. The SD-WAN edge 308 can include one or more real or virtual servers configured to execute one or more VNFs to perform the operations of the SD-WAN edge.

[0075] In this example, the SD-WAN edge 308 includes a bus 342 to couple hardware components of the hardware environment. The bus 342 couples a network interface card (NIC) 330, a storage disk 346, and one or more microprocessors 310 (hereinafter “microprocessors 310”). In some cases, a frontside bus can couple the microprocessors 310 and a memory device 344. In some examples, the bus 342 can couple the memory device 344, the microprocessors 310, and the NIC 330. The bus 342 can represent a peripheral component interface (PCI) express (PCIe) bus. In some examples, a direct memory access (DMA) controller can control DMA transfers between components coupled to the bus 342. In some instances, components coupled to the bus 342 control DMA transfers between components coupled to the bus 342.

[0076] The microprocessors 310 can include one or more processors each including independent execution units to execute instructions conforming to an instruction-set architecture stored to a storage medium. The execution units can be implemented as separate integrated circuits (ICs) or can be combined within one or more multi-core processors (or “many-core” processors), each processor implemented using a single IC (i.e., a chip multiprocessor).

[0077] The disk 346 represents a computer-readable storage medium including volatile and / or non-volatile, removable and / or non-removable media implemented in any method or technology for storage of information such as processor-readable instructions, data structures, program modules or other data. The computer-readable storage medium includes, but is not limited to, random access memory (RAM), read only memory (ROM), EEPROM, flash memory, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by the microprocessors 310.

[0078] The main memory 344 includes one or more computer-readable storage media that can include random access memory (RAM) such as various forms of dynamic RAM (DRAM), for example, DDR2 / DDR3 SDRAM, or static RAM (SRAM), flash memory, or any other form of fixed or removable storage media that is used to carry or store desired program code in the form of instructions or data structures and that is accessed by a computer. The main memory 344 provides a physical address space consisting of addressable memory locations.

[0079] The network interface card (NIC) 330 includes one or more interfaces 332 configured to exchange packets using links of an underlying physical network. The interfaces 332 can include port interface cards having one or more network ports. The NIC 330 can also include on-card memory, for example, to store packet data. Direct memory access transfers between the NIC 330 and other devices coupled to the bus 342 can read from / write to the NIC memory. The interfaces 332 can be interfaces used for the underlying connections of the WAN links configured for the SD-WAN module 306 between the SD-WAN edge 308 and one or more other SD-WAN edges.

[0080] The memory 344, the NIC 330, the storage disk 346, and the microprocessor 310 can provide an operating environment for a software stack that includes an operating system kernel 314 executing in kernel space. The kernel 314 can represent, for example, Linux, Berkeley Software Distribution (BSD), other Unix variant kernels, or a Windows Server operating system kernel available from Microsoft Corp. In some cases, the operating system can execute a hypervisor and one or more virtual machines managed by the hypervisor. Example hypervisors include Kernel-based Virtual Machine (KVM) for Linux kernels, Xen provided by VMware, ESXi, and Windows Hyper-V provided by Microsoft, among other open source and proprietary hypervisors. The term hypervisor can include a virtual machine manager (VMM). The operating system including the kernel 314 provides an execution environment for one or more processes in the user space 345. The kernel 314 includes physical drivers 325 that use the NIC 330.

[0081] The hardware environment and kernel 314 provide a user space 345 operating environment for the SD-WAN edge 308 modules, including the routing process 328, the configuration interface 374, and the SD-WAN module 306. The configuration interface 374 enables the SD-WAN controller 104 or operator to configure the SD-WAN edge 308. The configuration interface 374 can provide a NETCONF interface, a simple network management protocol (SNMP), a command line interface, a RESTful interface, a remote procedure call, or other interface by which a remote device can configure the SD-WAN edge 308 with configuration information stored to a configuration database 375. The configuration information can include, for example, SLA rules 322 that partially define the operation of the WAN link switching module 350 for the SD-WAN module 306, routes, and virtual routing and forwarding instances (VRFs) configured with interfaces for WAN links, link types (IP, MPLS, mobile, etc.) specified, priorities, maximum bandwidths, encapsulation information, overlay tunnel types, and / or other link characteristics.

[0082] The routing process 328 executes routing protocols to exchange routing information (e.g., routes) with other network devices and uses the routing information collected in the routing table 316 to select active routes to each destination, which are the routes used by the SD-WAN edge 308 to forward incoming packets to the destination. To route traffic from a source host to a destination host via the SD-WAN edge 308, the SD-WAN edge 308 learns the path to be taken by the packet. These active routes are inserted into a forwarding table 318 of the SD-WAN edge 308 and used by the forwarding plane hardware for packet forwarding. For example, the routing process 328 can generate the forwarding table 318 in the form of a radix or other lookup tree to map packet information (e.g., header information with destination information and / or label stack) to a next hop and ultimately to an interface 332 for output. In some examples, the SD-WAN edge 308 can have a physically bifurcated control plane and data plane, with a switch control card managing one or more packet forwarding line cards, each with one or more high-speed packet processors.

[0083] The SD-WAN edge 308 executes the SD-WAN module 306 to implement SD-WAN services, such as Figure 1The SD-WAN module 306 causes the SD-WAN edge 308 to forward traffic based on application flows. The SD-WAN module 306 uses packet characteristics to identify packets of different application flow packets. Once an application is identified using an initial packet, information for identifying traffic of an application session can be stored in a flow table for faster processing. The WAN link switching module 350 selects WAN links to assign applications according to routing information, policy information, performance data, and service characteristics of the WAN links for the SD-WAN service implemented by the SD-WAN module 306. The SD-WAN module 306 can program the forwarding table 318 with the selected WAN links for the applications, flow table data, or other data for mapping application traffic to the selected WAN links.

[0084] The SD-WAN edge 308 executes an AMR module 320. The AMR module 320 can include SLA rules 322, criteria (C) 323, and AMR priorities (AP) 324.

[0085] According to the techniques of the present disclosure, the AMR module 320 can be configured to identify one or more applications for AMR based on the criteria 323 for the SD-WAN system 100 Figure 1 ) where each of the criteria 323 is associated with a corresponding attribute of an application, each criterion having a weight associated therewith. The AMR module 320 can be configured to determine a violation of one of the SLAs on each WAN link associated with a first application of the one or more applications, and in response to determining the violation, apply AMR to the first application. Thus, a network device such as the SD-WAN edge 308 using the AMR module 320 can automatically identify an application for which AMR should be applied, and apply AMR to the application.

[0086] The AMR module 320 can maintain a dynamic list, table, or other data structure of prioritized applications for AMR (e.g., in the priorities of the AMR priorities 324) based on application attributes or characteristics and / or live traffic patterns in the network. The AMR module 320 can continuously analyze traffic in the network and build a list of applications for which AMR is desired or needed, e.g., based on the criteria 323. Once the AMR module 320 detects a SLA violation for any application for which QoE is relatively important (e.g., an application on the list of AMR applications in the AMR priorities 324), the AMR module 320 can apply AMR to the application, thereby ensuring QoE or enhancing QoE for those applications for which QoE is relatively important. In some examples, the AMR module 320 can also continuously monitor the SLAs of these identified applications for further remediation, e.g., disabling AMR when the SLA is again satisfied on at least one link associated with the application.

[0087] The AMR module 320 can dynamically identify relatively important applications and apply AMR to the application when the associated SLA is violated on all existing WAN links associated with the application. The AMR module 320 can also reverse the action (e.g., restore the previous operation of the application) when one or more WAN links are restored. For example, the AMR module 320 can identify relatively important applications and differentiate their priorities based on the criteria 323 and store the priorities in AMR priorities 324. The AMR module 320 can monitor the SLAs 322 and apply AMR for applications that violate the SLAs on all associated WAN links. For example, the AMR module 320 can monitor all WAN links and confirm whether there are sufficient resources (CPU and bandwidth) available for the identified applications. In some examples, the AMR module 320 can further apply AMR to the applications based on the priorities of the applications.

[0088] For example, the AMR module 320 can identify relatively important applications based on the criteria 323 and, in some examples, prioritize the relatively important applications. In some examples, the criteria 323 can include predetermined or default criteria programmed by the manufacturer of the SD-WAN edge 308. For example, the predetermined or default criteria can specify attributes or characteristics of an application that can be a group of applications with common attributes or characteristics. In some examples, there can be multiple default criteria stored in the criteria 323. For example, there can be different sets of criteria for different deployment environments, such as one for deployment in a bank network, one for deployment in an office network, etc. In some examples, some or all of the criteria in a set of criteria can be different from the criteria in another set of criteria. For more than one set of criteria, the criteria can have associated weights that are different in different sets of criteria or the same in different sets of criteria. For example, the weight for a bank can be 10 in one set of criteria and 2 in another set of criteria.

[0089] The following Table 1 is an example of default criteria for the criteria 323 that the AMR module 320 can use to identify applications for possible AMR. In this example, a weight is assigned to each criterion within a scale of 1 to 10, where 10 is most important for using AMR (typically an application that desires a higher QoE), however, any scale can be used.

[0090] Table 1 - Example Default Criteria for Identifying Applications for AMR

[0091] Application criteria Weight (scale 1-10, 10 being preferred) Based on RTP 10 Office applications 9 Applications with data packet sizes < 256 bytes 8

[0092] In the example of Table 1, the AMR module 320 can assign a weight of 10 to RTP-based applications, a weight of 9 to office-related applications (e.g., word processors, spreadsheet applications, presentation applications, etc.), and a weight of 8 to applications with small packet sizes (< 256 bytes). While certain criteria and certain weights are described with respect to Table 1, these are set forth by way of example, and any criteria or weights can be used in accordance with the techniques of this disclosure.

[0093] In some examples, a network administrator can choose to modify the default criteria for identifying applications for AMR and / or the associated weights of the criteria 323, or in the absence of default criteria, the network administrator can create criteria and / or weights based on the network administrator’s own usage. In some examples, if the network administrator modifies the default criteria or weights, this can override the default criteria or weights. For example, the AMR module 320 can overwrite the default criteria or weights in the criteria 323 with the modified criteria or weights. In other examples, the modified criteria or weights can be stored as a separate set of criteria, can be activated at a later time, and the original default criteria can be deactivated, but remain stored in the criteria 323. Table 2 is an example of custom criteria created or modified by a network administrator.

[0094] Table 2 - Example of Custom Criteria for Identifying Applications for AMR

[0095]

[0096] In the example of Table 2, the AMR module 320 can assign a weight of 4 to RTP-based applications, a weight of 5 to office-related applications, a weight of 8 to applications with small packet sizes (e.g., < 256 bytes), a weight of 9 to banking and / or trading (e.g., trading stocks, bonds, or other instruments) applications, a weight of 8 to streaming media applications, and a weight of 10 to applications with Differentiated Services Code Point (DSCP) markings. While certain criteria and certain weights are described with respect to Table 2, these are set forth by way of example, and any criteria or weights can be used.

[0097] For example, the AMR module 320 can monitor different applications with traffic in the SD-WAN system 100 Figure 1 ) and compare the different applications to each of the criteria of the criteria 323. In some examples, when a SLA breach occurs, there can not be enough network resources to apply AMR to every relatively important application. To address this example, the AMR module 320 can determine a priority for each application identified for AMR based on the weight given for each particular criterion met for that application. The AMR module 320 can store the respective priorities in the AMR priorities 324.

[0098] Figure 3 is a table diagram illustrating an example of prioritizing applications based on criteria. In the example of Figure 3 , the criteria are depicted in the top row, with each of the criteria having an associated weight shown in the second row. For example, RTP is a criterion with a weight of 4. Various applications are listed in the left column, such as a first video conferencing application. An "X" in the table indicates that the application meets the specified criterion. For example, the first video conferencing application meets the following criteria: RTP, office related, packet size less than 256 bytes, and streaming media. The absence of an "X" in the table indicates that the application does not meet the criterion. For example, the first video conferencing application does not meet the bank / transaction or DSCP marking criteria. The sum of each weight for each criterion that is met for each application is also shown. For example, the weight sum for the first video conferencing application is 25. The AMR module 320 Figure 2 may determine the cumulative weight of each criterion having a corresponding attribute to each of the applications identified for the AMR. The AMR module 320 can use the cumulative weight of each application to determine a priority for each application. In the example of Figure 3 , the first video conferencing application has the highest weight (25) and, therefore, is assigned the highest priority (shown as 1).

[0099] The AMR module 320 can determine the sum of the weights of the criteria that are met to determine a priority for each of the identified applications. For example, as Figure 3As shown, the applications having application traffic on the SD-WAN system 100 can include a first video conferencing application, a second video conferencing, an email application, a stock application, a banking application, an education application, a video application, and an FTP application. The first video conferencing application meets the criteria of the RTP application, the office application, having packets of less than 256 bytes, and the streaming media application. The AMR module 320 can add the weights associated with each of the criteria met by the first video conferencing application, resulting in a total weight of 25. In this example, the weight of 25 is the highest weight, which causes the AMR module 320 to assign the highest priority (priority 1) to the first video conferencing application. The AMR module 320 can add the weights associated with each of the identified applications in a similar manner. As such, in this example, the second video conferencing application, the stock application, and the banking application each have a total weight of 17, causing these applications to be priority 2. The email and education applications in this example have a total weight of 5, causing these applications to be priority 3. In this example, the video application and the FTP application do not meet any of the criteria (shown as not matching). As such, the AMR module 320 can identify the video application and the FTP application as not needing AMR or not being worth the resources associated with applying AMR to the applications. For example, the AMR module 320 can assign a priority of 0 to the applications to indicate that AMR should not be considered for these applications even if the associated SLAs for any of the applications are violated on all of the associated WAN links. As such, the AMR module 320 can not apply AMR to the video application or the FTP application even if all of the WAN links associated with the applications fail. In some examples, the AMR module 320 can store a table, list, or other data structure or entries in the AMR priority 324 indicating the priority of each application.

[0100] The AMR module 320 can proactively monitor the applications having a priority assigned in the AMR priority 324 for any SLA violations or breaches. In some examples, the AMR module 320 can not monitor any SLA violations or breaches for any applications assigned a priority of 0. In other examples, the AMR module 320 can also monitor any SLA violations or breaches for applications assigned a priority of 0.

[0101] SLA metrics can include jitter, latency, packet loss, bandwidth, or other SLA metrics and can be customized for each application. When an application identified in AMR priority 324 fails to meet its SLA on available WAN links, AMR module 320 can detect this and apply AMR to that application. In some examples, AMR module 320 can determine whether available resources, such as central processing unit (CPU) and link bandwidth, are sufficient and apply AMR to the application only if sufficient resources are available. For example, once AMR module 320 determines that SLA violations have occurred on all associated links for a particular application, it can perform a resource check before applying AMR to that application. As part of the resource check, system parameters such as CPU and link bandwidth can be defined and evaluated to avoid congestion and overload conditions in the SD-WAN system 100. In some examples, AMR module 320 can apply modifications to available resources when determining availability, such as 90%, 80%, or 70% of link bandwidth, or use predetermined thresholds to reserve some resources for other purposes, such as application traffic from new applications.

[0102] In some examples, if an SLA violation occurs for more than one application, the AMR module 320 can further determine whether to apply AMR based on the priority assigned to the applications. For example, if both the first and second video conferencing applications experience SLA violations on all WAN links, and there are only sufficient resources available to apply AMR to one of the two applications, the AMR module 320 can apply AMR to the first video conferencing application but not to the second video conferencing application because the first video conferencing application has a higher priority than the second video conferencing application.

[0103] Figure 4 This is a tabular diagram showing an example of applying AMR. Figure 4 In the example, the first video conferencing application (with priority 1—the highest priority) experiences SLA violations on both WAN link 1 and WAN link 2, as indicated by the "X" in the WAN link 1 and WAN link 2 columns. Thus, the AMR module 320 can apply AMR to the first video conferencing application. Other applications identified for AMR may only experience SLA violations on either WAN link 1 or WAN link 2, as indicated by the "X" in the corresponding column. Therefore, the AMR module 320 may not apply AMR to those applications, even if those applications have been identified as eligible for AMR. In this example, the video application and the FTP application are not eligible for AMR because they do not meet any criteria (indicated as mismatch).

[0104] The AMR module 320 can monitor and determine SLA violations, perform resource checks, and apply AMR to the first video conferencing application based on the determined SLA violations and sufficient resources being available to support the AMR for the first video conferencing application. The AMR module 320 can continue to monitor the SLA status and disable the AMR for the first video conferencing application when its SLA is again satisfied for the first video conferencing application. In this way, the AMR module 320 can release network resources supporting the AMR for the first video conferencing application that are no longer needed to satisfy the SLA for the first video conferencing application.

[0105] Figure 5 is a flowchart illustrating an example of automatically identifying applications for AMR and applying AMR techniques. A network device manufacturer, network administrator, or the AMR module 320 can define criteria for identifying important applications (400). For example, the network device manufacturer, network administrator, or the AMR module 320 can define criteria and can assign a weight to each criterion. Each of the criteria can be associated with a corresponding attribute of an application, e.g., an RTP application, an office-related application, an application with a small packet size, a banking / transaction application, a streaming media application, an application with a DSCP marking, etc. In some examples, the criteria are predetermined. In some examples, the criteria and / or weights are dynamic and the AMR module 320 can autonomously (i.e., without input from a network operator, customer, or other human agent) modify the criteria and / or weights based on the types of application traffic being experienced at a given time, network resources, network conditions, deployment topology, deployment environment, or type of business of the network operator or SD-WAN service customer, etc. For example, the AMR module 320 can add a new criterion to the criteria or remove a criterion from the criteria. For example, for a deployment in a banking network, the AMR module 320 can automatically remove one or more criteria that are less important to the operation of a banking business or add a banking criterion if the banking criterion is not already included in the default or predetermined criteria. In some examples, rather than or in addition to modifying the criteria, the AMR module 320 can modify one or more of the weights associated with the criteria. For example, the AMR module 320 can increase or decrease a weight associated with a criterion or add a weight for a new criterion. For example, for a deployment in a banking network, the AMR module 320 can increase a weight associated with a banking criterion and / or a weight associated with a small packet size. In some examples, at least a portion of the above operations can be performed by an SD-WAN controller, which then configures the SD-WAN module 306 with the criteria and / or weights of the criteria 323.

[0106] The AMR module 320 can determine whether any criteria are met for an application (402). For example, the AMR module 320 can compare the criteria to the properties of the application to determine whether any of the criteria are met for the application. If no criteria are met for the application (NO path from block 402), the AMR module 320 can ignore the application for AMR (406). For example, the AMR module 320 can determine that the application does not require AMR, or that the cost of applying AMR to the application exceeds the benefit of applying AMR to the particular application. In some examples, the AMR module 320 can give the application a priority of 0.

[0107] If any criteria are met for the application (YES path from block 402), the AMR module 320 can prioritize the application (404). For example, the AMR module 320 can add together the weights associated with each of the criteria met by the application to arrive at a total weight. The AMR module 320 can compare the total weight of the application to the total weights of other applications to determine a priority of the application. For example, if the total weight associated with the current application is the highest of all applications, the AMR module 320 can assign the highest priority, e.g., priority 1, to the current application. If the total weight associated with the current application is the lowest of all applications, the AMR module 320 can assign the lowest priority of those applications eligible for AMR to the current application.

[0108] The AMR module 320 can monitor the application and / or the WAN link associated therewith to monitor for any violations of the SLA associated with the application (408). For example, the AMR module 320 can monitor the performance (e.g., bandwidth, jitter, latency, etc.) of the WAN link associated with the application.

[0109] The AMR module 320 can determine whether the SLA is violated and whether resources are sufficient to apply AMR to the application (410). For example, the AMR module 320 can compare the performance to the SLA rules 322 to determine whether the SLA is violated. The AMR module 320 can determine resource availability and compare the resource availability to the expected resource consumption for applying AMR to the application to determine whether resources are sufficient. In some examples, when determining whether resources are sufficient to apply AMR to the application, the AMR module 320 can apply a modification to the amount of available resources, such as a percentage, to reserve a predetermined amount of network resources for other purposes, e.g., handling traffic from new applications.

[0110] If AMR module 320 determines that an SLA has been violated and there are sufficient resources to apply AMR to the application (from the "Yes" path in box 410), then AMR module 320 can apply AMR to the application (412). For example, AMR module 320 can copy application packets and can do so on more than two WAN links (e.g., Figure 1 On each WAN link (links 142N-A and 142N-N), a copy of each application packet is forwarded to the receiving network device. If the AMR module 320 determines that the SLA is not violated, resources are insufficient, or the SLA is not violated and resources are insufficient, the AMR module 320 may not apply the AMR and return to monitoring the application and / or WAN link (408). In some examples, if the SLA is violated, resources are insufficient, and the application has a higher priority than another application to which AMR has been applied, the AMR module 320 may stop applying AMR to the lower priority application, and if resources are sufficient to apply AMR to the higher priority application, the AMR module 320 may apply AMR to the higher priority application.

[0111] Figure 6 This is a flowchart illustrating an AMR automatic identification application and another example of applying AMR technology. For example, AMR module 320 ( Figure 2 AMR can identify one or more applications in SD-WAN based on standards, where each standard in the standard is associated with a corresponding attribute of the application (500). For example, AMR module 320 can associate one or more applications with standard 323 ( Figure 2 The AMR module 320 compares the applications to identify one or more applications for which an AMR may be applied. For example, the AMR module 320 may identify an application as eligible for an AMR based on the application having at least one attribute that matches at least one criterion. The AMR module 320 may also identify an application as ineligible for an AMR based on the application not having at least one attribute that matches at least one criterion.

[0112] The AMR module 320 can determine a SLA violation of the first application on each WAN link associated with the identified first application of the one or more applications (502). For example, the AMR module 320 can determine that a performance (e.g., bandwidth, latency, jitter, or other performance metric) of each WAN link associated with the first application is below a requirement of a SLA associated with the first application. In some examples, each WAN link can violate the SLA in the same manner, e.g., each WAN link can violate the SLA due to low bandwidth. In some examples, each WAN link can violate the SLA in a different manner, e.g., one WAN link can violate the SLA due to low bandwidth while another WAN link can violate the SLA due to high latency. In some examples, some WAN links can violate the SLA in the same manner while one or more WAN links can violate the SLA in a different manner.

[0113] The AMR module 320 can apply the AMR to the first application in response to determining the violation (504). For example, the AMR module 320 can duplicate application data packets of the first application and forward a copy of each application data packet on each WAN link of the at least two WAN links.

[0114] In some examples, the AMR is not pre-provisioned for the first application. For example, a network administrator does not manually provision the AMR for the first application.

[0115] In some examples, the criteria are predetermined and include at least one of RTP, office related, size, bank, transaction, or DSCP marking. In some examples, the AMR module 320 can autonomously modify the criteria. In some examples, each of the criteria has an associated weight and the AMR module 320 can autonomously modify one or more of the weights associated with the criteria.

[0116] In some examples, the AMR module 320 can determine that available resources of the SD-WAN edge 308 are sufficient to support the AMR of the first application, where applying the AMR to the first application is based on the available resources being sufficient to support the AMR of the first application.

[0117] In some examples, the identified one or more applications includes a second application. In some examples, each of the criteria has an associated weight. In some examples, the AMR module 320 can determine a first priority of the first application and a second priority of the second application based on the criteria, the first priority being higher than the second priority. In some examples, the AMR module 320 can determine a violation of one of the one or more SLAs associated with the second application. In this example, the AMR module 320 can determine that the available resources are insufficient to support AMR for both the first application and the second application. In this example, the AMR module 320 can enforce AMR for the first application and refrain from enforcing AMR for the second application based on the first priority being higher than the second priority.

[0118] In some examples, the first priority is based on a cumulative weight of each criterion having an attribute corresponding to the first application, and the second priority is based on a cumulative weight of each criterion having an attribute corresponding to the second application.

[0119] In some examples, the AMR module 320 is configured to transmit a copy of a data packet associated with the first application via each of the two or more WAN links.

[0120] In some instances, the techniques of the disclosure are performed by the SD-WAN edge 308. In some examples, identifying one or more applications and determining a violation of a SLA for AMR are performed by the SD-WAN controller 104 Figure 1 ) and the SD-WAN controller 104 outputs configuration data to cause the SD-WAN edge 308 to enforce AMR for the first application.

[0121] The techniques of the disclosure have several potential advantages over other techniques. For example, the techniques of the disclosure provide dynamic identification of relatively important applications. The techniques of the disclosure facilitate enabling and disabling AMR for a given application on demand. The techniques of the disclosure can provide better application QoE for relatively important applications. The techniques of the disclosure can provide better management and efficient utilization of WAN links and system resources. The techniques of the disclosure can reduce the burden on network administrators in determining whether AMR should be provisioned for a given application and in manually provisioning AMR for such applications.

[0122] The techniques described herein can be implemented in hardware, software, firmware, or any combination thereof. Various features described as modules, units or components can be implemented together in an integrated logic device, or separately as discrete but interoperable logic devices or other hardware devices. In some cases, various features of electronic circuitry can be implemented as one or more integrated circuit devices, such as integrated circuit chips or chipsets.

[0123] If implemented in hardware, the present disclosure can be directed to an apparatus such as a processor or an integrated circuit device, such as a chip or a chipset, for example. Alternatively or additionally, if implemented in software or firmware, the techniques can be realized at least in part by a computer-readable data storage medium comprising instructions that, when executed by a processor, perform one or more of the methods described above. For example, the computer-readable data storage medium can store such instructions for execution by a processor.

[0124] A computer readable medium can form part of a computer program product, which can include packaging material. The computer readable medium can include a computer data storage medium such as random access memory (RAM), read only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read only memory (EEPROM), FLASH memory, magnetic or optical data storage media, and the like. In some examples, the article of manufacture can include one or more computer readable storage media.

[0125] In some examples, the computer readable storage media can include non-transitory media. The term "non-transitory" can indicate that the storage medium is not among the transient signals that change over time, such as the waves of carrier or propagating signals. In some examples, a non-transitory storage medium can store data that can change over time, such as in RAM or buffer memory.

[0126] Code or instructions can be software and / or firmware executed by processing circuitry, which comprises one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term "processor" as used herein can refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein can be provided within software modules or hardware modules within the devices described herein.

Claims

1. A network device, comprising: The storage is configured to store information associated with one or more Service Level Agreements (SLAs) for applications in Software-Defined Wide Area Networks (SD-WAN). as well as Application-based multipath routing (AMR) module, including processing circuitry, is configured to: The first and second applications are identified based on standards for application-based multipath routing (AMR), wherein each standard in the standards is associated with a corresponding attribute of the application and has an associated weight. Identify a violation of one of the Service Level Agreements (SLAs) on each WAN link associated with the first application; Determine a violation of one or more Service Level Agreements (SLAs) on each WAN link associated with the second application; Based on the aforementioned standard, a first priority of the first application and a second priority of the second application are determined, wherein the first priority is higher than the second priority; It was determined that the available resources were insufficient to support application-based multipath routing (AMR) for both the first and second applications; and Based on the fact that the first priority is higher than the second priority, application-based multipath routing (AMR) is applied to the first application, and application-based multipath routing (AMR) is avoided from being applied to the second application.

2. The network device according to claim 1, wherein, There is no application-based multipath routing (AMR) pre-configured for the first application.

3. The network device according to claim 1, wherein, The standard is predetermined.

4. The network device according to claim 1, wherein, The application-based multipath routing (AMR) module is also configured to autonomously modify the standard.

5. The network device according to claim 1, wherein, Each of the standards has an associated weight, and the application-based multipath routing (AMR) module is also configured to autonomously modify one or more weights associated with the standard.

6. The network device according to any one of claims 1 to 5, wherein, The application-based multipath routing (AMR) module is further configured to: determine that the available resources of the network device are sufficient to support the application-based multipath routing (AMR) for the first application. The application of application-based multipath routing (AMR) to the first application is based on the premise that the available resources are sufficient to support application-based multipath routing (AMR) for the first application.

7. The network device according to claim 1, wherein, The first priority is based on the cumulative weight of each standard having attributes corresponding to the first application, and the second priority is based on the cumulative weight of each standard having attributes corresponding to the second application.

8. The network device according to any one of claims 1 to 5, wherein, To apply application-based multipath routing (AMR) to the first application, the application-based multipath routing (AMR) module is configured as follows: A copy of the data packet associated with the first application is transmitted via each of two or more WAN links.

9. A method for operating a network device, comprising: The processing circuitry identifies a first application and a second application for application-based multipath routing (AMR) in Software Defined Wide Area Network (SD-WAN) based on standards, wherein each standard in the standards is associated with a corresponding attribute of the application and has an associated weight. The processing circuitry determines a violation of one of the Service Level Agreements (SLAs) on each WAN link associated with the first application; The processing circuitry determines a violation of one or more Service Level Agreements (SLAs) on each WAN link associated with the second application; The processing circuit determines a first priority of the first application and a second priority of the second application based on the standard, wherein the first priority is higher than the second priority; The processing circuitry determines that available resources are insufficient to support application-based multipath routing (AMR) for both the first and second applications; and The processing circuit applies application-based multipath routing (AMR) to the first application based on the first priority being higher than the second priority, and avoids applying application-based multipath routing (AMR) to the second application.

10. The method according to claim 9, wherein, There is no application-based multipath routing (AMR) pre-configured for the first application.

11. The method according to claim 9, wherein, The standard is predetermined.

12. The method of claim 9, further comprising: The processing circuit can autonomously modify the standard.

13. The method according to claim 9, wherein, Each standard in the criteria has an associated weight, and the method further includes: The processing circuitry autonomously modifies one or more weights associated with the standard.

14. The method according to any one of claims 9 to 13, further comprising: The processing circuitry determines that the available resources of the network device are sufficient to support application-based multipath routing (AMR) for the first application. The application of application-based multipath routing (AMR) to the first application is based on the premise that the available resources are sufficient to support application-based multipath routing (AMR) for the first application.

15. The method according to claim 9, wherein, The first priority is based on the cumulative weight of each standard having attributes corresponding to the first application, and the second priority is based on the cumulative weight of each standard having attributes corresponding to the second application.

16. The method according to any one of claims 9 to 13, wherein, The method is performed by the SD-WAN edge device.

17. The method according to any one of claims 9 to 13, wherein, The SD-WAN controller performs an application-based multipath routing (AMR) to identify one or more applications and determine service level agreement (SLA) violations, wherein the SD-WAN controller outputs configuration data to cause the SD-WAN edge device to apply the application-based multipath routing (AMR) to the first application.

18. A computer-readable storage medium having instructions encoded therein for causing one or more programmable processors to perform the method described by any one of claims 9 to 17.

Images