On-demand probing for quality of experience metrics

By dynamically adjusting the detection granularity in SD-WAN devices, the problem of resource waste when link changes frequently is solved, more efficient experience quality evaluation and SLA satisfaction are achieved, and network performance is improved. Extensibility.

CN113676361BActive Publication Date: 2024-11-01HEWLETT PACKARD ENTERPRISE DEV LP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111153706.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-03-11
Filing Date
2019-06-27
Publication Date
2024-11-01
Estimated Expiration
2039-06-27

AI Technical Summary

Technical Problem

Existing software-defined wide area network (SD-WAN) devices need to continuously detect all WAN links when evaluating quality of experience (QoE), resulting in waste of resources and degradation of performance, especially in environments where link changes frequently. , it is difficult to efficiently meet the service level agreement (SLA) requirements for different applications.

Method used

The SD-WAN device initiates the detection process only on the link that currently receives traffic, stops the detection of the previous link, and starts on the new link, sends the detection packet only through the queue being used to dynamically adjust the detection granularity, as needed Send detection packets to reduce unnecessary detection processes.

Benefits of technology

It improves the scalability of the SD-WAN network, reduces the consumption of computing resources, reduces bandwidth and processing time, reduces equipment load and improves efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113676361B_ABST
    Figure CN113676361B_ABST
Patent Text Reader

Abstract

Embodiments of the present disclosure relate to on-demand detection for quality of experience metrics. In general, the present disclosure describes techniques for evaluating the quality of experience metrics of an application via a software-defined wide area network. For example, a network device may receive an application data packet for a data stream of an application. In response to receiving the application data packet, the network device may assign the data stream to a first link of a plurality of links and initiate a detection process for the data stream on the first link to determine one or more quality of experience (QoE) metrics for the first link. Later, the network device may detect that the data stream is no longer received. In response to detecting that the data stream is no longer received, the network device may stop the detection process for the data stream on the first link.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application of the Chinese invention patent application with national application number 201910567325.0, application date June 27, 2019, and invention name “On-demand detection for experience quality measurement”. Technical Field

[0002] The present disclosure relates to computer networks. Background Art

[0003] Over the past few decades, the Internet has grown exponentially from a small network including a small number of nodes to a global pervasive network that provides services to more than one billion users. Today, individual users are not limited to running a few network sessions with voice and / or data downloads on the network. On the contrary, the range of services used by users varies greatly in terms of multimedia games, audio and video streaming, network services, voice over IP (VoIP), etc. With the penetration of new technologies, such as the increase in the utilization of Internet of Things (IoT) and M2M (machine to machine) communications, the network services and software applications that a given user may need also change from a few sessions to multiple sessions with concurrent streams. As users increasingly run multiple applications, services, and transactions simultaneously, this number is growing rapidly. The increase in the number and variety of user sessions and packet flows has brought challenges to network service providers in terms of network performance (such as delay, latency, and jitter). Summary of the invention

[0004] In general, the present disclosure describes techniques for evaluating traffic for a particular application and initiating a corresponding synthetic probe process when traffic for the particular application is received, wherein the probe process generates various quality of experience (QoE) metrics for the link over which the traffic is transmitted. Similarly, when a software defined wide area network (SD-WAN) device implementing the techniques described herein detects that traffic for a particular application has completed, or the SD-WAN device is not receiving traffic, the SD-WAN device can stop the synthetic probe process corresponding to the application.

[0005] In some SD-WANs, the SD-WAN routing devices can specify paths for data flows between client devices and application servers. These paths are typically selected using service level agreement (SLA) parameters and various metrics of the WAN links. While SLA parameters can be more static in nature, or at least predetermined before the SD-WAN device receives the flow, the metrics of the various WAN links can be more dynamic, as the metrics describing the capabilities of a particular WAN link can vary based on various current aspects of the network. These metrics are obtained by sending probe packets on the various links and analyzing the transmission results, where the probe packets, which have the same size as the data packets in the data flow, reasonably measure how a particular WAN link handles the data flow.

[0006] Instead of sending one or more pre-configured probe packets over each WAN link to continuously provide all necessary QoE metrics for each link, the techniques described herein may initiate a probe process only on the link that is currently receiving traffic. To elaborate on the benefits of these techniques, the SD-WAN devices described herein may utilize only one or more subsets of those probe packets that will provide valuable QoE metrics for specific applications associated with the received and transmitted traffic, rather than sending a set of pre-configured probe packets as any traffic flows over the link.

[0007] If in the middle of a traffic transmission, the traffic must be moved to a different link, the SD-WAN device described in this article can stop the probing process for the previous link and initiate the probing process for the new link. Traffic may change links for various reasons, including switching from the default WAN link to the best equal-cost multi-path (ECMP) link based on the application's QoE metrics and SLA metrics. Another reason can be that the original link no longer meets the application's SLA metrics, but the SLA metrics can be met by moving the traffic to the second link.

[0008] These techniques can also be extended within the queues of the corresponding links. For example, a single WAN link can include multiple different queues. Instead of sending one or more probe packets via each queue of the WAN link, the SD-WAN device can send one or more probe packets only via a specific queue that is experiencing traffic. This additional level of granularity can increase the benefits of the techniques described herein.

[0009] There may be one or more advantages to using the techniques described herein. As an example, because the techniques of the present disclosure can send probe packets as needed, an SD-WAN network implementing the techniques described herein can be more scalable, enabling such a network to include more potential links and more client devices without a corresponding increase in the overhead caused by the probe process. For a system with the same number of links and client devices, an SD-WAN implementing the techniques described herein can consume less computing resources, such as bandwidth, memory, and battery power, than previous systems. This can generally improve the efficiency of the SD-WAN system because an SD-WAN implementing the techniques described herein can consume less bandwidth and reduce processing time for generating QoE metrics. In general, the techniques described herein can reduce the load on SD-WAN devices by reducing the amount of storage and processing of service level agreement (SLA) results, logs, and the like.

[0010] In one example of the technology described herein, a method is described, the method including, in response to receiving an application data packet for a data flow of an application by a network device, allocating the data flow by a network device to a first link among a plurality of links. The method also includes, in response to receiving the application data packet for the data flow, initiating, by the network device, a probing process for the data flow on the first link to determine one or more quality of experience (QoE) metrics for the first link. The method also includes detecting, by the network device, that the network device no longer receives the data flow. The method also includes, in response to detecting that the network device no longer receives the data flow, the network device stopping the probing process for the data flow on the first link.

[0011] In another example of the technology described herein, a network device is described. The network device includes a memory. The network device also includes one or more processors in communication with the memory. The one or more processors are configured to, in response to receiving an application data packet for a data stream of an application, assign the data stream to a first link of a plurality of links. The one or more processors are also configured to, in response to receiving an application data packet for the data stream, initiate a detection process for the data stream on the first link to determine one or more quality of experience (QoE) metrics for the first link. The one or more processors are also configured to detect that the network device no longer receives the data stream. The one or more processors are also configured to stop the detection process for the data stream on the first link in response to detecting that the network device no longer receives the data stream.

[0012] In another example of the technology described herein, a non-transitory computer-readable storage medium is described that stores instructions thereon that, when executed, cause one or more processors to: in response to receiving an application data packet of a data flow of an application, assign the data flow to a first link of a plurality of links. The instructions also cause the one or more processors to initiate a detection process for the data flow on the first link in response to receiving an application data packet of the data flow to determine one or more quality of experience (QoE) metrics of the first link. The instructions also cause the one or more processors to detect that the network device is no longer receiving the data flow. The instructions also cause the one or more processors to stop the detection process for the data flow on the first link in response to detecting that the network device is no longer receiving the data flow.

[0013] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a block diagram illustrating an example software defined wide area network system that performs an on-demand active synthetic sounding process in accordance with techniques of this disclosure.

[0015] Figure 2 is a block diagram illustrating an example software defined wide area network in accordance with one or more techniques of this disclosure.

[0016] Figure 3 is a block diagram illustrating an example software defined wide area network device configured to perform one or more on-demand proactive synthetic probing functions using a traffic engine in accordance with techniques of this disclosure.

[0017] Figure 4 is a flow chart illustrating an example technique for a software defined wide area network system for performing on-demand proactive synthetic probing functionality in accordance with techniques of this disclosure. DETAILED DESCRIPTION

[0018] Figure 1 is a block diagram illustrating an example software defined wide area network system that performs an on-demand active synthetic sounding process in accordance with techniques of this disclosure.

[0019] Figure 1 The example network system of includes a service provider network 2 that operates as a private network to provide packet-based network services to a user device 16. That is, the service provider network 2 provides authentication and establishment of network access for the user device 16 so that the user device can begin exchanging data packets with a public network 12, which may be an internal or external packet-based network such as the Internet.

[0020] exist Figure 1 In an example of , the service provider network 2 includes an access network 6 that provides connectivity to a public network 12 via a service provider software defined wide area network 7 (hereinafter referred to as "SD-WAN 7") and a router 8. The SD-WAN 7 and the public network 12 provide packet-based services that can be requested and used by user devices 16. As an example, the SD-WAN 7 and / or the public network 12 can provide large amounts of data transmission, Internet Protocol Voice (VoIP), Internet Protocol Television (IPTV), Short Message Service (SMS), Wireless Application Protocol (WAP) services, or customer-specific application services. The public network 12 can include, for example, a local area network (LAN), a wide area network (WAN), the Internet, a virtual LAN (VLAN), an enterprise LAN, a layer 3 virtual private network (VPN), an Internet Protocol (IP) intranet operated by a service provider operating the access network 6, an enterprise IP network, or some combination thereof. In various examples, the public network 12 is connected to a public WAN, the Internet, or other networks. The public network 12 implements one or more packet data protocols (PDPs), such as IP (IPv4 and / or IPv6), X.25, or Point-to-Point Protocol (PPP), to enable packet-based transmission services of the public network 12 .

[0021] Typically, user devices 16 are connected to gateway router 8 via access network 6 to receive connectivity for user services for applications hosted by user devices 16 or router 9. For example, a user may represent an enterprise, a residential user, or a mobile user. For example, user device 16 may be a personal computer, a laptop, or other type of computing device located behind customer equipment (CE) 11, which may provide local routing and switching functions. Each of user devices 16 may run various software applications, such as word processing and other office support software, web browsing software, software supporting voice calls, video games, video conferencing, and email, etc. For example, user devices 16 may be various network-enabled devices, commonly referred to as "Internet of Things" (IoT) devices, such as cameras, sensors (S), televisions, appliances, etc. In addition, user devices 16 may include mobile devices that access data services of service provider network 2 via radio access network (RAN) 6. Example mobile user devices include mobile phones, laptops or desktop computers with, for example, 3G wireless network cards, wireless-enabled netbooks, video game devices, pagers, smart phones, personal data assistants (PDAs), etc.

[0022] The network service provider operates or in some cases leases elements of the access network 6 to provide packet transport between the user device 16 and the router 8. The access network 6 represents a network that aggregates data traffic from one or more user devices 16 for transmission to / from the service provider's SD-WAN 7. The access network 6 includes network nodes that execute communication protocols to transmit control and user data to facilitate communication between the user device 16 and the router 8. The access network 6 may include a broadband access network, a wireless LAN, a public switched telephone network (PSTN), a customer premises equipment (CPE) network, or other types of access networks, and may include or otherwise provide connectivity to a cellular access network such as a radio access network (RAN) (not shown). Examples include networks that conform to the Universal Mobile Telecommunications System (UMTS) architecture, an evolution of UMTS known as Long Term Evolution (LTE), mobile IP standardized by the Internet Engineering Task Force (IETF), and other standards proposed by the Third Generation Partnership Project (3GPP), the Third Generation Partnership Project 2 (3GGP / 2), and the WiMAX Forum.

[0023] SD-WAN device 18 may be a customer edge (CE) router, a provider edge (PE) router, or other network device between access network 6 and SD-WAN 7. SD-WAN 7 provides packet-based connectivity to user devices 16 attached to access network 6 for access to public network 12 (e.g., the Internet). SD-WAN 7 may represent a public network owned and operated by a service provider that interconnects multiple networks (which may include access network 6). SD-WAN 7 may implement multi-protocol label switching (MPLS) forwarding and in such instances may be referred to as an MPLS network or MPLS backbone. In some instances, SD-WAN 7 represents multiple interconnected autonomous systems, such as the Internet that provide services from one or more service providers. Public network 12 may represent the Internet. Public network 12 may represent an edge network coupled to SD-WAN 7 via a transit network 22 and one or more network devices (e.g., customer edge devices, such as customer edge switches or routers). Public network 12 may include a data center. Router 8 may exchange packets with service node 10 via virtual network 20 , and router 8 may forward packets to public network 12 via transit network 22 .

[0024] In the example of network 2 including a wired / broadband access network, router 8 may represent a broadband network gateway (BNG), a broadband remote access server (BRAS), an MPLS PE router, a core router or gateway, or a cable modem termination system (CMTS). In the example of network 2 including a cellular access network as access network 6, router 8 may represent a mobile gateway, such as a gateway general packet radio service (GPRS) service node (GGSN), an access gateway (aGW), or a packet data network (PDN) gateway (PGW). In other examples, the functions described with respect to router 8 may be implemented in a switch, a service card, or another network element or component. In some examples, router 8 itself may be a service node.

[0025] A network service provider managing at least a portion of the network 2 typically provides services to users associated with devices (e.g., user devices 16) that access the service provider network 2. For example, the services provided may include traditional Internet access, VoIP, video and multimedia services, and security services. As described above with respect to the SD-WAN 7, the SD-WAN 7 may support multiple types of access network infrastructures that connect to a service provider network access gateway to provide access to the provided services. In some instances, the network system may include user devices 16 attached to multiple different access networks 6 having varying architectures.

[0026] Typically, any one or more of the user devices 16 may request authorization and data services by sending a session request to a gateway device such as an SD-WAN device 18 or router 8. In turn, the software defined wide area network (“SD-WAN”) device 18 may access a central server (not shown), injecting an authentication, authorization, and accounting (AAA) server, to authenticate one of the user devices 16 requesting network access. After authentication, any of the user devices 16 may send user data traffic to the SD-WAN 7 to access and receive services provided by the public network 12, and such packets may traverse the router 8 as part of at least one packet flow. In some examples, the SD-WAN device 18 may forward all authenticated user traffic to the public network 12, and if the user traffic requires services on the service node 10, the router 8 may apply the service 15 and / or direct the specific user traffic to the data center 9. The application (e.g., service application) to be applied to the user traffic may be hosted on the service node 10.

[0027] For example, when forwarding user traffic, the router 8 can guide individual user packet flows through services 15 executed on one or more service cards installed in the router 9. Additionally or alternatively, the service provider network 2 includes a data center 9, which has a set of service nodes 10 that provide an execution environment for most virtualized network services. In some examples, each of the service nodes 10 represents a service instance. Each of the service nodes 10 can apply one or more services to traffic. Therefore, the router 8 can guide user packet flows through a defined set of services provided by the service nodes 10. That is, in some examples, each user packet flow can be forwarded through a specific ordered combination of services provided by the service nodes 10, and each ordered set is referred to as a "service chain" herein. As examples, service 15 and / or service node 10 may apply stateful firewall (SFW) and security services, deep packet inspection (DPI), carrier-grade network address translation (CGNAT), traffic destination function (TDF) services, media (voice / video) optimization, Internet Protocol Security (IPSec) / Virtual Private Network (VPN) services, Hypertext Transfer Protocol (HTTP) filtering, counting, billing, charging, and / or load balancing of packet flows, or other types of services applied to network traffic.

[0028] exist Figure 1 In the example of , user packet flows may be directed along a service chain including service 15 and / or any of the services applied by service node 10. After processing at the terminal node of the service chain (i.e., the last service to be applied to a packet flowing along a particular service path), the traffic may be directed to public network 12.

[0029] While a "service chain" defines one or more services applied in a specific order to provide a composite service of an application to a packet bound to the service chain, a "service tunnel" or "service path" refers to the logical and / or physical path taken by the packet flow processed by the service chain and the forwarding state of the packet flow forwarded according to the service chain order. Each service chain can be associated with a corresponding service tunnel, and the packet flow associated with each user device 16 flows along the service tunnel according to the service profile associated with the corresponding user. For example, a given user can be associated with a specific service profile, which is in turn mapped to a service tunnel associated with a specific service chain. Similarly, another user can be associated with a different service profile, which is in turn mapped to a service tunnel associated with a different service chain. In some examples, after the SD-WAN 18 authenticates the user and establishes an access session, the SD-WAN 18 or router 8 can guide the packet flow for the user along the appropriate service tunnel, so that the data center 9 applies the necessary customized services (ordered services) for the given user. In some examples, the SDN controller 14 can also provide a forwarding rule set to the SD-WAN 18 or router 8 for managing the forwarding path. In some examples, SDN controller 14 manages the forwarding path through all elements in data center 9 starting with router 8 .

[0030] In some examples, the service nodes 10 may implement the service chain using internally configured forwarding state that directs packets of a packet flow along the service chain for processing according to an identified set of service nodes 10. Such forwarding state may specify a tunnel interface for tunneling between service nodes 10 using network tunnels such as IP or Generic Routing Encapsulation (GRE) tunnels, network virtualization using GRE (NVGRE), or by using VLANs, Virtual Extensible LANs (VXLANs), MPLS technology, etc. In some instances, actual or virtual switches, routers, or other network elements interconnecting the service nodes 10 may be configured to direct packet flows to the service nodes 10 according to the service chain.

[0031] exist Figure 1In the example of , the service provider network 2 includes a software defined network (SDN) and a network function virtualization (NFV) architecture. The SDN controller device 14 may provide a high-level controller for configuring and managing the routing and switching infrastructure of the service provider network 2. The NFV orchestrator device 13 may provide a high-level orchestrator for configuring and managing the virtualization of network services in the service node 10 of the data center 9. In some instances, the SDN controller 14 manages the deployment of virtual machines (VMs) within the operating environment of the data center 9. For example, the SDN controller 14 may interact with the provider edge (PE) router 8 to specify service chain information described in more detail below. For example, the service chain information provided by the SDN controller 14 may specify any combination and order of services provided by the service node 10, traffic engineering information for tunneling or otherwise transmitting packet flows along a service path, rate limiting, service type (TOS) marking, or packet classifiers that specify criteria for matching packet flows to specific service chains. Further example details of an SDN controller are described in PCT International Patent Application PCT / US13 / 44378 filed on June 5, 2013, the entire contents of which are incorporated herein by reference.

[0032] Although illustrated as part of the data center 9, the service node 10 can be a network device coupled by one or more switches or virtual switches of the SD-WAN 7. In one example, each of the service nodes 10 can run as a VM in a virtual computing environment. In addition, the computing environment can include a scalable cluster of general-purpose computing devices such as servers based on x86 processors. As another example, the service node 10 may include a combination of general-purpose computing devices and dedicated devices. As a virtualized network service, a single network service provided by the service node 10 can be expanded as in a modern data center by configuring virtualized memory, processor utilization, storage and network policies, and horizontally by adding additional load-balanced VMs. In other examples, the service node 10 can be a gateway device or other router. In other examples, the functions described with respect to each of the service nodes 10 can be implemented in a switch, service card, or other network element or component.

[0033] As described herein, elements within the network system 2, such as the SD-WAN device 18, perform application data monitoring, for example, using various application quality of experience (QoE) metric functions, such as real-time performance monitoring (RPM) or two-way active measurement protocol (TWAMP). For example, the active measurement protocol (TWAMP). That is, RPM and TWAMP can be used within the service provider network 2 to measure one-way and two-way or round-trip metrics of network performance on a per-user basis between network devices (also referred to as hosts or endpoints), such as path connectivity, path delay, packet jitter, packet loss, packet reordering, etc. Typically, the QoE measurement architecture includes network devices, each of which supports the protocol used and performs a specific role to initiate a data session and exchange test packets for the data session. In Figure 1 In the example network architecture illustrated in FIG, the SD-WAN device 18 is configured to perform QoE metric prediction. The SD-WAN device 18 allows load sharing across connections and adjusts traffic based on network conditions to improve performance.

[0034] In an SD-WAN topology, there may be many "hubs and spokes" that include user devices 16 and services 15 that are connected to each other. In order to measure the QoE metrics of each of these connections, the SD-WAN device 18 can control the transmission of probe packets that are designed to measure these QoE metrics based on the transmission process across each of the corresponding connections (also called paths or links). However, all paths may not be experiencing traffic at a given time, which means that continuing to transmit probe packets on these links may provide limited benefits and only result in link performance degradation or unnecessary use of central processing units (CPUs) on the devices to transmit and receive links. The probe packets can have an actual size similar to the size of actual application traffic, which means that the probe packets can consume considerable bandwidth when transmitted. The techniques described herein can simplify the process so that the probe process is only completed on paths that are experiencing traffic, rather than continuously performing synthetic probes on all those paths. Once the traffic on a particular path stops, the probe process will also stop.

[0035] The SD-WAN device 18 executing the path selection algorithm also determines QoE metrics, such as service level agreement (SLA) metrics including round trip time (RTT), jitter, and packet loss, which are affected by the real-time parameters of the application, such as packet size, queues, and packet bursts used to determine the best path. However, different applications have different packet sizes in their data flows. In addition, when links in the network are not transmitting data flows for a particular application received from the service 15 and router 8, if active probing packets corresponding to the particular application are transmitted on the same links to determine the SLA metrics, there may be unnecessary traffic on these links, which may consume valuable bandwidth that can be used for other data flows. As described below with reference to Figure 2 and Figure 3 As described, the techniques described herein illustrate how the SD-WAN device 18 can implement a machine learning algorithm to determine when traffic for a particular application is received, match that traffic to a specific configuration of a probe packet, and begin a synthetic probe process that is active only while traffic for that particular application is continuously being received.

[0036] According to the techniques described herein, the SD-WAN device 18 may not perform a probing process on a first link of a plurality of links. After a period of time, the SD-WAN device 18 may receive an application data packet of a data stream of an application. In response to receiving the application data packet, the SD-WAN device 18 may assign the data stream to the first link and initiate a probing process for the data stream on the first link to determine one or more QoE metrics for the first link. After the data stream is completed or otherwise interrupted such that the SD-WAN device 18 no longer receives the data stream, the SD-WAN device 18 may detect that the data stream is no longer received. In response to detecting that the data stream is no longer received, the SD-WAN device 18 may stop the probing process for the data stream on the first link. The probing process described herein may include sending a preconfigured group of probing packets on a link, regardless of the type of traffic received. In other instances, the probing process described herein may include transmitting only a single probe packet or smaller groups of two or more probe packets specifically designed to collect QoE metrics applicable to applications transmitting data streams, the QoE metrics being based on various parameters such as packet size in the data stream, the presence or size of bursts within the data stream, the packet size of the probe packets, and the burstiness of the probe packets. A bursty stream, as measured by burstiness, involves data packets that are sent intermittently in bursts, rather than a continuous stream.

[0037] In some examples, the metrics carried by the QoE probe packets transmitted during the probing process may include one or more of a timestamp for sending or receiving a test packet, an error estimate for sending or receiving a test packet, a sequence number for sending a test packet, a time to live (TTL) value for a test packet, a count of keep-alive packet data units (PDUs) and / or service packets, bytes, or users. One-way and two-way network performance measurements may include keep-alive or path connectivity, round-trip time (RTT), path delay, packet jitter, packet reordering, packet loss, service delay measurements, or service load measurements based on the received metrics.

[0038] Figure 2 is a block diagram illustrating an example SD-WAN 37 in accordance with one or more techniques of this disclosure. In the examples described herein, the SD-WAN 7 includes three different WAN links: a first WAN link 40 coupling the SD-WAN device 38 to a multi-protocol layer switching (MPLS) network 50, a second WAN link 42 coupling the SD-WAN device 38 to the Internet 52, and a third WAN link 44 coupling the SD-WAN device 38 to a long term evolution (LTE) network 54. In other examples, the SD-WAN 7 may include any number of links of any suitable type for transporting data flows between a client side (e.g., client devices 36 and SD-WAN devices 38) and an application side (e.g., SD-WAN devices 56 and application servers 58).

[0039] According to the techniques described herein, the SD-WAN device 38 may receive application data packet traffic for a particular application. For example, the client device 36 may begin executing the application locally. The execution of the application may include communicating with the application server 58, and the SD-WAN device 38 may be configured to forward the data stream from the client device 36 to the application server 58 via any one of the links 40, 42, and 44.

[0040] Upon receiving a data stream for an application from a client device 36, the SD-WAN device 38 may assign the data stream to one of the links 40, 42, 44. The SD-WAN device 38 may perform this assignment based on whether the link is the default link for the system, the link is the default link for the application, or the link is the best link at the time based on the QoE metrics of each of the links. For example, the SD-WAN device 38 may assign the data stream to the Internet link 42. In addition to assigning the data stream, the SD-WAN device 38 may also initiate a probing process for the Internet link 42 by beginning to send one or more probing packets on the Internet link 42 to determine one or more QoE metrics of the Internet link 42.

[0041] The SD-WAN device 38 may detect at a later time, by completion or some other form of interruption, that traffic for a particular application has ended. In response to this determination, the SD-WAN device 38 may stop the synthetic probe process by refraining from sending additional probes corresponding to the particular application.

[0042] In some instances, the SD-WAN device 38 may perform these techniques on multiple queues or overlay paths in each of the WAN links 40, 42, and 44. For example, the Internet link 42 may include 8 different queues or overlay links that can be used to process data flows. Instead of generally sending multiple probe packets to the Internet link 42, the SD-WAN device 38 may send probe packets for a specific probing process only on queues within the Internet link 42 that are actually transmitting data flows to obtain metrics for queues in the Internet link 42 in a more fine-grained sense. Throughout this disclosure, any technique that can be performed on a per-link basis may also be performed on a per-queue basis within the corresponding link, including reallocating data flows to other queues within the same link, or reallocating data flows from queues in one link to queues in a different link.

[0043] The SD-WAN device 38 can configure a single default overlay link until traffic is received. After the traffic is transmitted to a specific node N1, the node N1 can notify the SD-WAN device 38, and the SD-WAN device 38 can obtain the configuration of all links for that ECMP (e.g., policies, synthetic probes, etc. suitable for the application). Active probing begins at this point. Before the SLA results of the synthetic probe are available, the traffic can be placed on the default link. Once the results are available, based on the SLA metrics, the traffic can be sent on the link that meets the SLA.

[0044] Figure 3 1 is a block diagram illustrating an example software defined wide area network device 80 configured to perform one or more on-demand active synthetic detection functions using a traffic engine 110 in accordance with the techniques of the present disclosure. The SD-WAN device 80 may be found herein at Figure 1 8 and may represent, for example, either a router 8 or an SD-WAN device 18. Additionally, although described with respect to a particular network device (e.g., a router), the techniques may be implemented by any network device that may operate as an SD-WAN device, such as a customer premises device, a layer 3 (L3) or L2 / L3 switch, or a server.

[0045] In this example, the SD-WAN device 80 is divided into three logical or physical "planes" to include a control plane 81 that performs control operations for the device, a data plane 85 for forwarding transit network traffic, and a service plane 83 for applying one or more network services 87 to transit packet flows forwarded by the router. That is, the router 81 implements three separate functions (e.g., routing / control, forwarding data, and network service functions) logically (e.g., as separate software instances executed on the same set of hardware components) or physically (e.g., as separate physical dedicated hardware components that statically implement functions in hardware or dynamically execute software or computer programs to implement functions). In this example, a high-speed internal switching fabric 105 couples the control plane 81, the service plane 83, and the data plane 85 to deliver data units and control messages between the units. The switching fabric 105 can represent an internal switching fabric or a crossbar, bus, or link.

[0046] exist Figure 3 In the example of , the control plane 81 includes a control unit 82 having a main microprocessor 102, which performs device management services, user authentication, and control plane routing functions of the SD-WAN device 80. The microprocessor 102 may include one or more general or special-purpose processors, such as a digital signal processor (DSP), an ASIC, a field programmable gate array (FPGA), or any other equivalent logic device. Therefore, the terms "processor" or "controller" used herein may refer to any one or more of the foregoing structures or any other structure that is operable to perform the techniques described herein. Executable programs such as traffic engines 110 and services 131 can be operated by the microprocessor 102 to perform various actions, operations, or functions of the SD-WAN device 80. For example, the microprocessor 102 of the SD-WAN device 80 can obtain and execute instructions stored by various data stores that cause the microprocessor 102 to perform operations of the traffic engine 110 and services 131.

[0047] One or more storage components within the SD-WAN device 80 (e.g., the RIB 104) may store information for processing during operation of the SD-WAN device 80 (e.g., the SD-WAN device 80 may store data accessed by the traffic engine 110 and services 131 during execution at the SD-WAN device 80). In some examples, the storage components are temporary storage, meaning that the primary purpose of the storage components is not long-term storage. The storage components on the SD-WAN device 80 may be configured as volatile memory to store information for a short period of time, and therefore do not retain the stored contents if power is lost. Examples of volatile memory include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and other forms of volatile memory known in the art.

[0048] In some examples, the storage component also includes one or more computer-readable storage media. The storage component in some examples includes one or more non-transitory computer-readable storage media. The storage component can be configured to store a larger amount of information than is typically stored by volatile memory. The storage component can also be configured as a non-volatile storage space for long-term storage of information and retaining information after power on / off cycles. Examples of non-volatile memory include magnetic hard disks, optical disks, floppy disks, flash memory, or in the form of electrically programmable memory (EPROM) or electrically erasable and programmable (EEPROM) memory. The storage component can store program instructions and / or information (e.g., data) associated with the traffic engine 110 and the service 131. The storage component 248 may include a memory configured to store data or other information associated with the traffic engine 110 and the service 131.

[0049] In general, control unit 82 represents hardware or a combination of hardware and control software that implements control plane protocols 89A-89N ("routing protocols 89") to learn and maintain routing information within routing information base 104 ("RIB 104"). RIB 104 may include information that defines a network (such as Figure 1 The routing protocol 89 interacts with the kernel 100 executing on the control unit 82 (e.g., through API calls) to update the RIB 104 based on the routing protocol messages received by the SD-WAN device 80. The kernel 100 can parse the topology defined by the routing information in the RIB 104 to select or determine one or more routes through the network. For example, the kernel can generate forwarding information in the form of forwarding information bases 103A-103N ("FIB 103") based on the network topology represented in the RIB 104, that is, perform route resolution. Typically, the kernel 100 generates the FIB 103 in the form of a radix or other lookup tree to map packet information (e.g., header information with destination information and / or label stack) to the next hop and ultimately to the interface port of the IFC 114 associated with the corresponding forwarding unit 112. For example, each of the FIBs 103 can associate a network destination with a specific next hop and a corresponding IFC 114. For MPLS-related traffic forwarding, for a given FEC, FIB 103 stores label information including incoming labels, outgoing labels, and the next hop for the packet. The control unit 82 can then program the forwarding unit 112 of the data plane 85 using FIB 103, which the control unit 82 installs in the lookup ASIC 106.

[0050] In this example, the data plane 85 is a decentralized data plane in that the data plane functions and packet forwarding functions are distributed among a plurality of forwarding units 112A-112N (“forwarding units 112”). In the example of the SD-WAN device 80, the data plane 85 includes forwarding units 112 that provide high-speed forwarding of network traffic received by interface cards 114A-114N (“IFCs 44”) via inbound links 116A-116N to outbound links 118A-118N. The forwarding units 112 may each include one or more packet forwarding engines (“PFEs”) coupled to respective interface cards 114, and may represent, for example, a dense port concentrator (DPC), a modular port concentrator (MPC), a flexible physical interface card (PIC) concentrator (FPC), or other line cards, for example, that may be inserted into a chassis or combination of chassis of the SD-WAN device 80.

[0051] like Figure 3 As shown in the example of , each forwarding unit 112 includes a corresponding one of the lookup ASICs 106A-106N ("lookup ASICs 106") that receive control and data session traffic via the IFC card 114, perform route lookups, and based on the routes installed to the FIB 103, forward the traffic to the control unit 82 (control traffic destined for the SD-WAN device 80) or to one of the forwarding units 40 (transit data traffic) for output via an interface to one of the output links 48. In one example, the lookup ASICs 106 are microcode-controlled chipsets that are programmably configured by a slave microprocessor (not shown) executing on each of the forwarding units 112. Specifically, in this example, each of the ASICs 106 can be controlled by an internal microcode programmed from the microprocessor.

[0052] When forwarding a packet, the control logic within each lookup ASIC 106 traverses the corresponding FIB 103, and upon reaching a FIB entry (e.g., a leaf node) for the packet, the microcode-implemented control logic automatically selects one or more forwarding next hops (FNHs) for forwarding the packet. In this manner, as the packet traverses the internal architecture of the SD-WAN device 80, the ASICs 106 of the forwarding units 112 process the packet by performing a series of operations on each packet on the corresponding internal packet forwarding path. For example, operations may be performed on each packet based on any of the corresponding ingress interface, ingress PFE 114, egress PFE 114, egress interface, or other components of the SD-WAN device 80 to which the packet is directed prior to egress (such as one or more service cards). The forwarding units 112 each include a forwarding structure that, when executed, examines the contents of each packet (or attributes of another packet, e.g., an incoming interface) and, based thereon, makes forwarding decisions, applies filters, and / or performs billing, management, traffic analysis, and load balancing, for example.

[0053] In one example, each of the forwarding units 112 arranges the forwarding structure into next hop data that can be linked together as a series of "hops" along an internal packet forwarding path for a network device. In many instances, the forwarding structure performs a lookup operation within the internal memory of the ASIC 106, where the lookup can be performed for a tree (or dictionary tree) search, a table (or index) search. Other example operations that can be specified with a next hop include filter determination and application, or rate limiter determination and application. The lookup operation locates an item that matches the content of the packet or an item that matches another attribute of the packet or packet flow (such as the inbound interface of the packet) within a lookup data structure (e.g., a lookup tree). The way in which the packet is forwarded by the forwarding unit 112 from an input interface on one of its IFCs 114 to an output interface on one of its IFCs 114 or otherwise processed is determined based on the result of the packet processing of the operation defined by the next hop forwarding structure within the ASIC 106.

[0054] Lookup ASIC 106 may be implemented using a forwarding application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or any other equivalent integrated or discrete logic circuit device, as well as any combination of these components. Each of forwarding units 112 may include substantially similar components to perform substantially similar functions.

[0055] The service plane 83 of the SD-WAN device 80 includes a plurality of service units 113A-113K ("service units 13"), which may be, for example, removable service cards configured to apply network services to packets flowing through the data plane 85. That is, when forwarding a packet, the forwarding unit 112 may direct the packet to the service plane 83 to apply one or more network services 131 by the service unit 113. In this example, each of the service units 113 includes a microprocessor 127 configured to execute a hypervisor 129 to provide an operating environment for a plurality of network services 131. As examples, the service units 113 may apply firewall and security services, carrier-grade network address translation (CG-NAT), media optimization (voice / video), IPSec / VPN services, deep packet inspection (DPI), HTTP filtering, counting, billing, charging, and load balancing or other types of services applied to packet flows of network traffic. For example, each of the services 131 may be implemented as a virtual machine or container executed by the hypervisor 129 and the microprocessor 127.

[0056] exist Figure 3 In the example of , the control unit 82 provides an operating environment for the traffic engine 110. In some examples, the control unit 82 can use the traffic engine 110 to perform one or more TWAMP or RPM logical roles (such as controlling a client, a server, a session sender, and a session reflector).

[0057] If there are N nodes in the SD-WAN network including the SD-WAN device 80, each node can be connected to another node through multiple equal cost overlay paths called path groups. If M is the number of overlapping paths between any two given nodes, and P is the number of SLA probes with different probe types and / or probe parameters, the number of flows that need to be maintained under normal circumstances will be:

[0058] (N*(N-1) / 2)*M*P(1)

[0059] At a given time, if traffic is actually only sent on X links out of N*(N-1) / 2 total links. Therefore, the number of flows that need to be maintained by the SD-WAN device 80 is only:

[0060] X*M*P(2)

[0061] The traffic engine 110 may be configured to perform the techniques described throughout this disclosure. For example, according to the techniques described herein, the traffic engine 110 may not perform a detection process on a first link of the plurality of links. After a period of time, the traffic engine 110 may receive an application data packet of a data flow of the application.

[0062] In response to receiving the application data packet, the traffic engine 110 can assign the data flow to the first link and initiate a probing process for the data flow on the first link to determine one or more QoE metrics of the first link. In some instances, initiating the probing process for the data flow includes sending one or more synthetic probe packets on the first link to determine one or more QoE metrics of the first link. These synthetic probe packets can simply be a plurality of probe packets, each having a unique default configuration, and the plurality of probe packets can be independent of any specific characteristics of the application or data flow.

[0063] In other instances, in addition to the link-specific detection described throughout the present disclosure, the traffic engine 110 may also perform application-specific detection. For example, in response to the traffic engine 110 receiving an application data packet of a data stream of an application, the traffic engine 110 may determine an application signature of the application data packet, such as by using deep packet inspection (DPI). The traffic engine 110 may then determine a probe packet configuration for the data stream based on the application signature. The probe packet configuration may be such that the packet size and / or burstiness of the probe packet sent on the first link is similar to the packet size and / or burstiness of the application traffic processed by the traffic engine 110. The traffic engine 110 may determine this information in real time using the DPI, or may determine this information using the DPI, store an indication of the application signature and this information in a database, and then the traffic engine 110 may obtain this information when the application sends additional traffic later. In these instances, initiating a detection process for the data stream will include the traffic engine 110 sending one or more probe packets configured according to the probe packet configuration on the first link to determine one or more QoE metrics of the first link.

[0064] In some instances, the plurality of links may be a plurality of equal cost multi-path (ECMP) links. In these instances, initiating a probing process for a data flow may include, in response to receiving an application data packet of a data flow of an application, the traffic engine 110 determining an application signature of the application data packet, such as by using DPI. The traffic engine 110 may then determine a set of one or more service level agreement (SLA) metrics of the application and a destination of the data flow based on the application signature. When the traffic engine 110 determines the best link for the data flow toward the destination, the traffic engine 110 may assign the data flow to a default link of the plurality of ECMP links. The traffic engine 110 may then send one or more probe packets on each of the plurality of links to determine a corresponding QoE metric for each of the plurality of ECMP links. Now with the QoE metric and the SLA metric for each ECMP link, the traffic engine 110 may determine which ECMP link has a QoE metric that satisfies the SLA metric of the application. Traffic engine 110 may then reallocate the data flow from the default link to an ECMP link that meets the SLA metric of the application, refraining from sending additional probe packets on each of the plurality of ECMP links except the first link.

[0065] In some additional instances, the techniques described herein may be performed at a queue level rather than at an entire link level. For example, when a first link includes multiple queues, the traffic engine 110 may assign a data flow to the first link by assigning the data flow to a first queue of the multiple queues of the first link. When initiating a probe process for a data flow, the traffic engine 110 may send one or more probe packets that it would normally send on the entire link only on the first queue of the first link (rather than on each queue in the link) to determine one or more QoE metrics for the first queue.

[0066] After the data flow is completed or otherwise interrupted such that the traffic engine 110 is no longer receiving the data flow, the traffic engine 110 can detect that the data flow is no longer being received. To address possible momentary terminations in the transmission of data packets, in some instances, the traffic engine 110 can detect that data packets are no longer being received in the data flow and wait a threshold amount of time to determine whether any additional data packets are received. If additional data packets are received within the threshold amount of time, the traffic engine 110 can continue the probing process for the data flow on the first link. Otherwise, if no additional data packets are received after the threshold amount of time has passed, the traffic engine 110 can confirm that the data flow is no longer being received.

[0067] In response to detecting that the data flow is no longer being received, the traffic engine 110 can stop the detection process for the data flow on the first link. In some examples, stopping the detection process for the data flow includes refraining from sending additional synthetic detection packets on the first link, whether these packets are default packets or specifically configured data packets, and whether the detection packets are generally sent on the link or on a specific queue.

[0068] At some time after initiating the probing process for the data stream, other traffic may be received or some other event may occur, causing one or more QoE metrics of the first link to fail to meet the service level agreement (SLA) metrics of the application. After the traffic engine 110 determines that this is the case, the traffic engine 110 may send one or more probe packets on a second link of the multiple links to determine one or more QoE metrics of the second link. The traffic engine 110 may send the one or more probe packets on the multiple links one by one until a satisfactory link is found, or may send each of the one or more probe packets on each of the multiple links to the predetermined destination approximately simultaneously to simultaneously determine the QoE metrics of each link.

[0069] In any case, after the traffic engine 110 determines that one or more QoE metrics of the second link meet the SLA metrics of the application, the traffic engine 110 can reallocate the first data flow from the first link to the second link. The traffic engine 110 can then stop the detection process for the data flow on the first link and continue the detection process for the data flow on the second link.

[0070] The traffic engine 110 may process additional data streams in a similar manner, ensuring that the detection process only occurs on the link that is currently experiencing traffic. For example, the traffic engine 110 may receive a second application data packet of a second data stream of a second application while the first data stream is still being transmitted on the first link. In response to receiving the second application data packet, the traffic engine 110 may assign the second data stream to a second link of the plurality of links, the second link being different from the first link. The traffic engine 110 may then initiate a detection process for the second data stream on the second link to determine one or more QoE metrics for the second link. At this point, the traffic engine 110 will only detect two links in the plurality of links, rather than each link in the plurality of links. In a manner similar to the traffic received for the first link, the traffic engine 110 may detect that the network device is no longer receiving the second data stream. In response to detecting that the second data stream is no longer being received, the traffic engine 110 may stop the detection process for the second data stream on the second link.

[0071] An optional addition to the techniques described herein may be to run pre-configured synthetic probes appropriate for the application type. In some basic examples, the traffic engine 110 may run all pre-configured synthetic probes when transmitting traffic on a particular link.

[0072] In some other instances, the traffic engine 110 can perform on-demand probing techniques at the overlay path level, providing even more granularity to the system. The techniques described herein can help scale nodes in a mesh network. In an SD-WAN environment, these probes can be performed on an overlay that can be MPLS, GRE, IPSEC, or a combination of MPLS / GRE / IPSEC based.

[0073] The traffic engine 110 can continue this process in a loop, starting the detection process for the application when the application is received and assigned to a specific link, and stopping the detection of the link that no longer receives the application traffic. In this way, the traffic engine 110 can be more scalable, enabling such a network to include more potential links and more client devices without correspondingly increasing the overhead caused by the detection process. For a system with the same number of links and client devices, the traffic engine 110 consumes less computing resources on the SD-WAN than previous systems, such as bandwidth, memory, and battery power. This can generally improve the efficiency of the SD-WAN system because the traffic engine 110 can consume less bandwidth and reduce the processing time for generating QoE metrics. In general, the techniques described herein can reduce the load on the traffic engine 110 by reducing the processing of service level agreement (SLA) results, logs, and the like.

[0074] Figure 4 is a flow chart illustrating an example technique for a software defined wide area network system for performing on-demand active synthetic detection functionality according to the techniques of the present disclosure. The example operations may be performed by Figure 1 The traffic engine 110 executed on the SD-WAN device 18 is executed. The following are the steps of the process, but other examples of processes performed in the technology of the present disclosure may include additional steps or may not include some of the steps listed below.

[0075] According to the techniques described herein, the SD-WAN device 18 may not initially perform a probing process on a first link of the plurality of links. After a period of time, the traffic engine 110 may receive an application data packet for a data flow of an application (402). In response to receiving the application data packet, the traffic engine 110 may assign the data flow to the first link (404) and initiate a probing process for the data flow on the first link to determine one or more QoE metrics for the first link (406). After the data flow is completed or otherwise interrupted such that the SD-WAN device 18 is no longer receiving the data flow, the traffic engine 110 may detect that the data flow is no longer being received (408). In response to detecting that the data flow is no longer being received, the traffic engine 110 may stop the probing process for the data flow on the first link (410).

[0076] The detection process described herein may include sending a set of pre-configured detection packets on a link, regardless of the type of traffic received. In other instances, the detection process described herein may include transmitting only a single detection packet or a smaller group of two or more detection packets, which is specifically designed to collect QoE metrics for applications suitable for transmitting data streams, the QoE metrics are based on various parameters, such as the packet size in the data stream, the presence or burst size of bursts within the data stream, the packet size of the detection packets, and the burstiness of the detection packets. Burst flows measured by burstiness involve data packets that are sent intermittently in a bursty manner, rather than a continuous stream.

[0077] The techniques described herein may be implemented in hardware, software, firmware, or any combination thereof. The various features described as modules, units, or components may be implemented together in an integrated logic device, or separately as discrete but interoperable logic devices or other hardware devices. In some cases, the various features of an electronic circuit device may be implemented as one or more integrated circuit devices, such as an integrated circuit chip or chipset.

[0078] If implemented in hardware, the present disclosure may be directed to a device such as a processor or an integrated circuit device such as an integrated circuit chip or chipset. Alternatively or additionally, if implemented in software or firmware, the techniques may be implemented at least in part by a computer-readable data storage medium including instructions that, when executed, cause a processor to perform one or more of the above methods. For example, a computer-readable data storage medium may store such instructions for execution by a processor.

[0079] The computer-readable medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may include computer data storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic or optical data storage media, etc. In some examples, the article of manufacture may include one or more computer-readable storage media.

[0080] In some examples, computer-readable storage media may include non-transitory media. The term "non-transitory" may indicate that the storage medium is not implemented in a carrier wave or propagating signal. In some examples, non-transitory storage media may store data that may change over time (e.g., in RAM or cache).

[0081] The code or instructions may be software and / or firmware executed by a processing circuit device, which includes one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuit devices. Therefore, the term "processor" as used herein may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functions described in the present disclosure may be provided within a software module or a hardware module.

[0082] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. A method for communication, include: Allocating, by a network device, a data flow to a first queue of a plurality of queues associated with a first network link, for sending data packets of the data flow from the first queue to the first network link; Initiating, by the network device, probing the network link via the first queue based on the allocation to determine one or more quality of experience (QoE) metrics for the first network link measured via the first queue; Determining, by the network device, an application signature of an application data packet of the data flow; determining, by the network device based on the application signature and the probe, that the QoE metric does not satisfy an SLA metric for the application matching the application signature; reallocating, by the network device based on the determination that the QoE metric does not satisfy the SLA metric, the data flow from the first queue to a different second queue for sending subsequent data packets of the data flow from the second queue to a second network link; sending, by the network device, one or more probe packets on the second network link via the second queue based on the reallocation to determine one or more QoE metrics for the second network link measured via the second queue, the one or more probe packets being configured according to a probe packet configuration; as well as Additional probe packets configured according to the probe packet configuration are suppressed from being sent via the first queue on the first network link based on the reallocation by the network device.

2. The method according to claim 1, in, After refraining from sending additional probe packets via the first queue, the second queue is the only queue on which probe packets are currently being sent. The method of claim 1 , wherein each of the one or more probe packets has a distinct default configuration. The method of claim 3 , wherein the one or more probing packets comprise a plurality of composite probing packets.

5. The method according to claim 1, further comprising: include: The probe packet configuration for the data flow is determined by the network device based on the application signature.

6. The method of claim 1, wherein determining the application signature of the application data packet include: A deep packet inspection of the application data packets is performed by the network device.

7. A method according to any one of claims 1 to 6, wherein the data flow includes application data traffic for a first application, and wherein reallocating the data flow includes reallocating the data flow based on determining that: (1) one or more quality of experience metrics for the first queue associated with the first network link fail to meet a service level agreement for the first application, and (2) the one or more quality of experience metrics for the second queue associated with the second network link meet the service level agreement for the first application.

8. The method according to any one of claims 1 to 6, wherein the network device comprises a software defined wide area network (WAN) device, wherein the first network link comprises a first WAN link, and wherein the second network link comprises a second WAN link.

9. A network device, include: Memory; as well as one or more processors in communication with the memory, the one or more processors configured to: assigning the data flow to a first queue of a plurality of queues associated with the first network link; initiating probing of the first network link via the first queue based on the allocation to determine one or more quality of experience (QoE) metrics of the first queue for the first network link; determining an application signature of an application data packet of the data flow; determining, based on the application signature and the probe, that the QoE metric does not satisfy an SLA metric for the application matching the application signature; reallocating the data flow from the first queue to a different second queue for sending subsequent data packets of the data flow from the second queue to a second network link based on the determination that the QoE metric does not satisfy the SLA metric; sending one or more probe packets on the second network link via the second queue based on the reallocation to determine one or more QoE metrics for the second network link measured via the second queue, the one or more probe packets being configured according to a probe packet configuration; as well as Based on the reallocation, additional probe packets configured according to the probe packet configuration are refrained from being sent on the first network link via the first queue.

10. The network device of claim 9, wherein the second queue is the only queue on which probe packets are currently being sent based on the one or more processors having refrained from sending additional probe packets via the first queue.

11. The network device of claim 9, wherein each of the one or more probe packets has a distinct default configuration.

12. The network device of claim 11, wherein the one or more probe packets comprise a plurality of composite probe packets.

13. The network device of claim 9, wherein the one or more processors are further configured to: The probe packet configuration for the data flow is determined based on the application signature.

14. The network device of claim 9, wherein the one or more processors are configured to determine the application signature of the application data packet. include: The one or more processors are configured to perform deep packet inspection of the application data packets.

15. The network device according to any one of claims 9 to 14, wherein the data flow comprises application data traffic for a first application, and wherein the one or more processors are configured to redistribute the data flow include: The one or more processors are configured to reallocate the data flow based on a determination that: (1) one or more quality of experience metrics for the first queue associated with the first network link fail to meet a service level agreement for the first application, and (2) the one or more quality of experience metrics for the second queue associated with the second network link meet the service level agreement for the first application.

16. The network device according to any one of claims 9 to 15, wherein the network device comprises a software defined wide area network (WAN) device, wherein the first network link comprises a first WAN link, and wherein the second network link comprises a second WAN link.

17. A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors of a network device to: assigning the data flow to a first queue of a plurality of queues associated with the first network link; initiating probing of the first network link via the first queue based on the allocation to determine one or more quality of experience (QoE) metrics of the first queue for the first network link; determining an application signature of an application data packet of the data flow; determining, based on the application signature and the probe, that the QoE metric does not satisfy an SLA metric for the application matching the application signature; reallocating the data flow from the first queue to a different second queue for sending subsequent data packets of the data flow from the second queue to a second network link based on the determination that the QoE metric does not satisfy the SLA metric; sending one or more probe packets on the second network link via the second queue based on the reallocation to determine one or more QoE metrics for the second network link measured via the second queue, the one or more probe packets being configured according to a probe packet configuration; as well as Based on the reallocation, additional probe packets configured according to the probe packet configuration are refrained from being sent on the first network link via the first queue.

18. The computer-readable storage medium of claim 17, wherein the network device comprises a software-defined wide area network (WAN) device, wherein the first network link comprises a first WAN link, and wherein the second network link comprises a second WAN link.