Method and apparatus for network management

Abstract

The present disclosure relates to methods and devices for network management. In general, the present disclosure describes technologies for dynamic application service level agreement (SLA) metric generation, distribution, and intent-based SD-WAN link selection. For example, a network device may determine a metric associated with an application or application group. The network device may send the metric to a controller, and in response, may receive a recommended SLA metric associated with the application or application group from the controller. The network device may also calculate the intent-based SLA metric based on the recommended SLA metric, one or more characteristics of one or more links connected to the network device, and a user-configured intent model that defines a tolerance level for applying the recommended SLA metric. The network device may select a path for sending traffic from an application or application group based on the intent-based SLA metric.

Description

[0001] This application is a divisional application of the invention patent application with the application date of September 29, 2019, the Chinese national application number 201910936905.2, and the invention name “Method and device for network management”. Technical Field

[0002] The present disclosure relates to computer networks. Background Art

[0003] A computer network is a collection of interconnected computing devices that can exchange data and share resources. In a packet-based network such as the Internet, computing devices transmit data by dividing the data into variable-length blocks called packets, which are individually routed through the network from a source device to a destination device. The destination device extracts the data from the packets and assembles the data into its original form.

[0004] Over the past few decades, the Internet has grown exponentially, from a small network consisting of a few nodes to a global pervasive network serving more than one billion users. Today, individual subscribers are not limited to running a few network sessions with voice and / or data downloads over the network. Instead, the range of services used by subscribers covers multimedia games, audio and video streaming, web services, IP voice (VoIP), etc. With the penetration of new technologies, such as the increased utilization of the Internet of Things (IoT) and M2M (machine to machine) communications, the network services and software applications that a given subscriber may need have also evolved from a few sessions to multiple sessions with concurrent streams. This number is growing rapidly as subscribers increasingly run multiple applications, services, and transactions simultaneously. The increase in the number and variety of subscriber sessions and packet flows poses challenges to network service providers in terms of network performance (such as latency, delay, and jitter).

[0005] A user may want a service provider to provide a service with an acceptable level of quality, often referred to as quality of experience (QoE). QoE can be measured based on various metrics of a link, including latency, delay (inter-frame gap), jitter, packet loss, and / or throughput. A user may define an expected level for one or more metrics of QoE that the user expects in a service contract with a service provider, such as a service level agreement (SLA). SLA metrics are typically user-configurable values ​​and are derived through trial and error or benchmarking environments with user experience or real-world best application metrics. Summary of the invention

[0006] In general, the present disclosure describes techniques for dynamic application SLA metric generation, distribution, and intent-based SD-WAN link selection in a software-defined wide area network (SD-WAN). Network devices (e.g., routers) of an SD-WAN can specify paths (e.g., links) for data flows between client devices and application servers. These paths are typically selected using service level agreement (SLA) metrics for WAN links that are used to transmit business flows. While SLA metrics can be more static in nature, or at least predetermined before a network device receives a flow, metrics for various WAN links can be more dynamic because metrics describing the capabilities of a particular WAN link can change based on various current aspects of the network. These metrics are obtained by sending probe packets on various links and analyzing the results of the transmission, where probe packets having the same size as data packets in a data flow reasonably measure how a particular WAN link handles a data flow.

[0007] In one implementation, one or more SD-WAN network devices (referred to herein as "SD-WAN leaf nodes") implementing one or more aspects of the techniques described herein may each dynamically learn one or more metrics in real time or near real time for links that send application traffic and / or traffic for a group / class of applications (referred to herein as "application groups"), and send the learned metrics to a centralized controller (e.g., an SLA controller). The SLA controller may calculate an ideal user experience based on an aggregation of the metrics received from each SD-WAN leaf node. The ideal user experience calculated by the SLA controller may be referred to as a "recommended SLA metric". Each SD-WAN leaf node may receive recommended SLA metrics for each application and / or application group, and calculate intent-based SLA metrics to identify the optimal / best path for sending traffic for the application and / or application group.

[0008] The technology described herein can provide one or more technical advantages that can provide at least one practical application. For example, the technology described herein provides dynamic application and / or application group metric learning from real-world network deployments through all connected SD-WAN leaf nodes. Each SD-WAN leaf node applies the SLA metric of the best user experience from real customer application services (rather than from laboratory design / test environment application services). By periodically sending recommended SLA metrics per application and / or application group by the SLA controller, the administrator can be exempted from the heavy metric configuration work for each leaf node and application when changing the WAN link capacity. The automatic SLA metric calculation at the SLA controller can release the extensive product engineering work of the SD-WAN supplier, that is, statically calculating the metrics for the application and / or application group recommended to the customer. In addition, the SLA controller continuously learns and calculates the metrics, and recommends the SLA metrics that can be used for each application and / or application group. By dynamically learning and calculating the metrics and recommending the SLA metrics for each application and / or application group, the calculation of the SLA metrics can provide more accurate and current SLA metrics relative to the manually configured SLA metrics. The intent-based expected SLA metric calculation at the SD-WAN leaf node is adapted to the network interface conditions and the user-configured intent model on the corresponding node, which enables the user to further fine-tune the recommended SLA metrics from the SLA controller. In addition, one or more aspects of the described technology provide full network visibility about which links meet or do not meet the expected SLA metrics in the SD-WAN network. This enables the discovery of paths / links in the SD-WAN that do not meet SLA requirements, which can be used to provide instructions to resolve the problem (e.g., by upgrading the link).

[0009] In one example, a method includes receiving, by a controller, a metric of a link from a plurality of network devices of a software defined wide area network (SD-WAN), wherein the metric is associated with an application or application group. The method also includes calculating, by the controller, a recommended service level agreement (SLA) metric for the application or application group based on an aggregation of the metric received from each of the plurality of network devices. The method also includes sending, by the controller, the recommended SLA metric to each of the plurality of network devices, so that each of the plurality of network devices calculates an intent-based SLA metric for selecting a path for sending traffic from the application or application group.

[0010] In another example, a method includes determining, by a network device among a plurality of network devices of a software defined wide area network (SD-WAN), a metric of a link of a network device, wherein the metric is associated with an application or an application group. The method also includes sending the metric to a controller by the network device. The method also includes receiving, by the network device, a recommended service level agreement (SLA) metric associated with an application or an application group from a controller, wherein the recommended SLA metric is calculated based on an aggregation of metrics from each of a plurality of network devices, including the metric sent by the network device. In addition, the method includes calculating, by the network device, an intent-based SLA metric based on: a recommended SLA metric, one or more characteristics of one or more links connected to the network device, and a user-configured intent model defining a tolerance level for applying the recommended SLA metric. The method also includes selecting, by the network device, a path for sending a service from an application or an application group based on the intent-based SLA metric.

[0011] In another example, a network device in a plurality of network devices of a software defined wide area network (SD-WAN) includes a memory. The network device also includes one or more processors in communication with the memory, the one or more processors being configured to: determine a metric of a link of the network device, wherein the metric is associated with an application or an application group. The one or more processors of the network device are also configured to send the metric to the controller. The one or more processors of the network device are also configured to receive a recommended service level agreement (SLA) metric associated with the application or application group from the controller, wherein the recommended SLA metric is calculated based on an aggregation of metrics from each of the plurality of network devices, including the metric sent by the network device. The one or more processors of the network device are also configured to calculate an intent-based SLA metric based on the following items: a recommended SLA metric, one or more characteristics of one or more links connected to the network device, and a user-configured intent model defining a tolerance level for applying the recommended SLA metric. The one or more processors of the network device are also configured to select a path for sending a business from an application or an application group based on the intent-based SLA metric.

[0012] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a block diagram illustrating an example SD-WAN that performs dynamic application SLA metric generation, distribution, and intent-based SD-WAN link selection in accordance with one or more aspects of the techniques described in this disclosure;

[0014] Figure 2is a block diagram illustrating example dynamic application SLA metric generation and distribution of recommended SLA metrics for an SD-WAN in accordance with one or more aspects of the techniques described in this disclosure;

[0015] Figure 3 is a block diagram illustrating a logical example of intent-based SLA metric calculation according to one or more aspects of the techniques described herein;

[0016] Figure 4 is a block diagram illustrating an example technique for path selection based on intent-based SLA metric calculation for SD-WAN in accordance with one or more aspects of the techniques described herein;

[0017] Figure 5 is a block diagram illustrating an example SLA controller configured in accordance with one or more aspects of the techniques described herein;

[0018] Figure 6 is a block diagram illustrating an example network device configured in accordance with one or more aspects of the techniques described herein; and

[0019] Figure 7 is a flow diagram illustrating an example technique for SD-WAN to perform dynamic application SLA metric generation, distribution, and intent-based SD-WAN link selection in accordance with one or more aspects of the techniques described in this disclosure. DETAILED DESCRIPTION

[0020] Figure 1 is a block diagram illustrating an example SD-WAN that performs dynamic application SLA metric generation, distribution, and intent-based SD-WAN link selection in accordance with one or more aspects of the techniques described in this disclosure. Figure 1 In the example shown, the service provider network 2 operates as a private network to provide packet-based network services to the subscriber device 16. That is, the service provider network 2 provides authentication and establishment of network access for the subscriber device 16 so that the subscriber device can begin exchanging data packets with the public network 12, which may be a packet-based internal or external network such as the Internet.

[0021] The service provider network 2 includes an access network 6 that provides connectivity to a public network 12 via a service provider software defined wide area network 7 (hereinafter referred to as "SD-WAN 7") and a router 8. The SD-WAN 7 and the public network 12 provide packet-based services that can be requested and used by subscriber devices 16. For example, the SD-WAN 7 and / or the public network 12 can provide bulk data transfer, voice over Internet Protocol (VoIP), Internet Protocol television (IPTV), short message service (SMS), wireless application protocol (WAP) services, or customer-specific application services. The public network 12 can include, for example, a local area network (LAN), a wide area network (WAN), the Internet, a virtual LAN (VLAN), an enterprise LAN, a layer 3 virtual private network (VPN), an Internet Protocol (IP) intranet operated by a service provider operating the access network 6, an enterprise IP network, or some combination thereof. In various examples, the public network 12 is connected to a public WAN, the Internet, or other networks. Public network 12 implements one or more packet data protocols (PDPs), such as IP (IPv4 and / or IPv6), X.25, or Point-to-Point Protocol (PPP), to enable packet-based delivery of public network 12 services.

[0022] Typically, subscriber devices 16 are connected to a gateway router 8 via an access network 6 to receive a connection for subscriber services of applications hosted by a public network 12 or a data center 9. A subscriber may represent, for example, an enterprise, a residential subscriber, or a mobile subscriber. Subscriber devices 16 may be, for example, a personal computer, a laptop, or other types of computing devices located behind a customer equipment (CE) 11, which may provide local routing and switching functions. Each subscriber device 16 may run various software applications, such as word processing and other office support software, web browsing software, software supporting voice calls, video games, video conferencing, and email, etc. For example, subscriber devices 16 may be various network-enabled devices, commonly referred to as "Internet of Things" (IoT) devices, such as cameras, sensors (S), televisions, appliances, etc. In addition, subscriber devices 16 may include mobile devices that access data services of a service provider network 2 via a radio access network (RAN) 4. Example mobile subscriber devices include mobile phones, laptops or desktop computers with, for example, a 3G wireless card, netbooks with wireless capabilities, video game devices, pagers, smart phones, personal data assistants (PDAs), etc.

[0023] The network service provider operates or in some cases leases elements of the access network 6 to provide packet transport between the subscriber device 16 and the router 8. The access network 6 represents a network that aggregates data traffic from one or more subscriber devices 16 for transport to / from the service provider's SD-WAN 7. The access network 6 includes network nodes that execute communication protocols to transmit control and user data to facilitate communication between the subscriber device 16 and the router 8. The access network 6 may include a broadband access network, a wireless LAN, a public switched telephone network (PSTN), a customer premises equipment (CPE) network, or other types of access networks, and may include or otherwise provide connectivity to a cellular access network such as a radio access network (RAN) 4. Examples include networks that conform to the Universal Mobile Telecommunications System (UMTS) architecture, an evolution of UMTS called Long Term Evolution (LTE), Mobile IP standardized by the Internet Engineering Task Force (IETF), and other standards proposed by the Third Generation Partnership Project (3GPP), the Third Generation Partnership Project 2 (3GGP / 2), and the WiMAX Forum.

[0024] SD-WAN device 18 may be a customer edge (CE) router, a provider edge (PE) router, or other network device between access network 6 and SD-WAN 7. SD-WAN 7 provides packet-based connectivity to subscriber devices 16 attached to access network 6 for access to public network 12 (e.g., the Internet). SD-WAN 7 may represent a public network owned and operated by a service provider to interconnect multiple networks (which may include access network 6). In some examples, SD-WAN 7 may implement multi-protocol label switching (MPLS) forwarding, and in this case may be referred to as an MPLS network or MPLS backbone. In some cases, SD-WAN 7 represents multiple interconnected autonomous systems, such as the Internet, that provide services from one or more service providers. SD-WAN 7 may include network devices 13 that forward application traffic on WAN links within SD-WAN 7, including SD-WAN devices 18 and routers 8. Public network 12 may represent the Internet. The public network 12 may represent an edge network coupled to the SD-WAN 7 via a transport network 22 and one or more network devices (e.g., a customer edge device, such as a customer edge switch or router). The public network 12 may include a data center. The SD-WAN device 8 may exchange packets with the service node 10 via the virtual network 20, and the router 8 may forward the packets to the public network 12 via the transport network 22.

[0025] In the example of network 2 including a wired / broadband access network, router 8 may represent a broadband network gateway (BNG), a broadband remote access server (BRAS), an MPLS PE router, a core router or gateway, or a cable modem termination system (CMTS). In the example of network 2 including a cellular access network as access network 6, router 8 may represent a mobile gateway, such as a gateway general packet radio service (GPRS) service node (GGSN), an access gateway (aGW), or a packet data network (PDN) gateway (PGW). In other examples, the functions described with respect to router 8 may be implemented in a switch, a service card, or another network element or component. In some examples, router 8 itself may be a service node.

[0026] A network service provider that manages at least a portion of the network 2 typically provides services to subscribers associated with devices (e.g., subscriber devices 16) that access the service provider network 2. The services provided may include, for example, traditional Internet access, VoIP, video and multimedia services, and security services. As described above with respect to the SD-WAN 7, the SD-WAN 7 may support multiple types of access network infrastructures that connect to a service provider network access gateway to provide access to the provided services. In some cases, the network system may include subscriber devices 16 attached to multiple different access networks 6 having different architectures.

[0027] Typically, any one or more subscriber devices 16 may request authorization and data services by sending a session request to a gateway device such as an SD-WAN device 18 or router 8. In turn, the SD-WAN device 18 may access a central server (not shown), such as an authentication, authorization, and accounting (AAA) server, to authenticate one of the subscriber devices 16 requesting network access. Once authenticated, any subscriber device 16 may send subscriber data traffic to the SD-WAN 7 to access and receive services provided by the public network 12, and such packets may traverse the router 8 as part of at least one packet flow. In some examples, the SD-WAN device 18 may forward all authenticated subscriber traffic to the public network 12, and if the subscriber traffic requires services on the service node 10, the router 8 may apply the services and / or direct the specific subscriber traffic to the data center 9. Applications (e.g., service applications) to be applied to subscriber traffic may be hosted on the service node 10.

[0028] For example, when forwarding subscriber traffic, router 8 can direct individual subscriber packet flows through services executed on one or more service cards installed in data center 9. In addition, or alternatively, service provider network 2 includes a data center 9 having a service node cluster 10, which provides an execution environment for most virtualized network services. In some examples, each service node 10 represents a service instance. Each service node 10 can apply one or more services to a traffic flow. In this way, router 8 can direct subscriber packet flows through a defined set of services provided by service node 10. That is, in some examples, each subscriber packet flow can be forwarded through a specific ordered combination of services provided by service node 10, each ordered set being referred to herein as a "service chain." For example, the services and / or service nodes 10 may apply stateful firewall (SFW) and security services, deep packet inspection (DPI), carrier-grade network address translation (CGNAT), traffic destination function (TDF) services, media (voice / video) optimization, Internet Protocol Security (IPSec) / virtual private network (VPN) services, Hypertext Transfer Protocol (HTTP) filtering of packet flows, counting, accounting, billing and / or load balancing, or other types of services applied to network traffic.

[0029] exist Figure 1 In the example of , a subscriber packet flow can be directed along a service chain, which includes any services applied by the service node 10. Once processed at the terminal node of the service chain (i.e., the last service to be applied to packets flowing along a particular service path), the traffic can be directed to the public network 12.

[0030] A "service chain" defines one or more services to be applied in a specific order to provide a combined service applied to packet flows bound to the service chain, while a "service tunnel" or "service path" refers to the logical and / or physical path taken by the packet flow processed by the service chain and the forwarding state used to forward the packet flow according to the service chain ordering. Each service chain can be associated with a corresponding service tunnel, and the packet flow associated with each subscriber device 16 flows along the service tunnel according to the service profile associated with the corresponding subscriber. For example, a given subscriber can be associated with a specific service profile, which is in turn mapped to a service tunnel associated with a specific service chain. Similarly, another subscriber can be associated with a different service profile, which is in turn mapped to a service tunnel associated with a different service chain. In some examples, after the SD-WAN device 18 has authenticated and established an access session for the subscriber, the SD-WAN device 18 or router 8 can direct the subscriber's packet flow along the appropriate service tunnel, so that the data center 9 applies the necessary ordered services for the given subscriber. In some examples, a centralized controller (not shown) for software defined networks (e.g., an SDN controller) may also provide a set of forwarding rules for managing forwarding paths to the SD-WAN device 18 or router 8. In some examples, the SDN controller manages the forwarding path starting from the router 8 through all elements in the data center 9.

[0031] In some examples, the service nodes 10 may implement service chaining using internally configured forwarding state that directs packets of a packet flow along the service chain for processing according to an identified set of service nodes 10. Such forwarding state may specify a tunnel interface for tunneling between service nodes 10 using a network tunnel such as an IP or Generic Routing Encapsulation (GRE) tunnel, Network Virtualization using GRE (NVGRE) or by using VLANs, Virtual Extensible LANs (VXLANs), MPLS technology, etc. In some cases, real or virtual switches, routers, or other network elements interconnecting the service nodes 10 may be configured to direct packet flows to the service nodes 10 according to the service chain.

[0032] exist Figure 1 In the example of FIG. 1 , the service provider network 2 includes a software defined network (SDN) and a network function virtualization (NFV) architecture. The SDN controller may provide a high-level controller device for configuring and managing the routing and switching infrastructure of the service provider network 2. The NVV coordinator device ( Figure 1The SDN controller (not shown) may provide a high-level coordinator for configuring and managing virtualization of network services to the service nodes 10 of the data center 9. In some cases, the SDN controller manages the deployment of virtual machines (VMs) within the operating environment of the data center 9. For example, the SDN controller may interact with the provider edge (PE) router 8 to specify service chain information. For example, the service chain information provided by the SDN controller may specify any combination and ordering of services provided by the service node 10, traffic engineering information for tunneling or otherwise transmitting packet flows along a service path, rate limiting, type of service (TOS) marking, or a packet classifier that specifies criteria for matching packet flows to specific service chains. Further example details of the SDN controller are described in PCT International Patent Application PCT / US13 / 44378 filed on June 5, 2013, the entire contents of which are incorporated herein by reference.

[0033] Although shown as part of the data center 9, the service node 10 can be a network device coupled by one or more switches or virtual switches of the SD-WAN 7. In one example, each service node 10 can run as a VM in a virtual computing environment. In addition, the computing environment can include a scalable cluster of general-purpose computing devices, such as servers based on x86 processors. As another example, the service node 10 may include a combination of general-purpose computing devices and dedicated devices. As a virtualized network service, the various network services provided by the service node 10 can be horizontally expanded by allocating virtualized memory, processor utilization, storage and network policies, and by adding additional load balancing VMs, just like in a modern data center. In other examples, the service node 10 can be a gateway device or other router. In other examples, the functions described with respect to each service node 10 can be implemented in a switch, a service card, or another network element or component.

[0034] As described herein, elements within the network system 2 (such as the SD-WAN device 18) perform application data monitoring using various application quality of experience (QoE) measurement functions, such as real-time performance monitoring (RPM), one-way active measurement protocol (OWAMP), two-way active measurement protocol (TWAMP), active probing, passive probing, protocol deep packet inspection (DPI) engine, or other measurement protocols or technologies. That is, these measurement protocols or technologies can be used within the service provider network 2 to measure real-time or near real-time metrics of the WAN links of the SD-WAN 7, such as path connectivity, path delay, packet jitter, packet loss, packet reordering, etc., for example, between network devices (also known as hosts or endpoints) on a per-subscriber basis. Typically, the QoE measurement architecture includes network devices, each of which supports the protocol / technology used and performs a specific role to initiate a data session (otherwise referred to as a "test session") and exchange test packets for the data session. The test packets are exchanged with embedded metrics (such as timestamps) used to calculate the metrics.

[0035] In some cases, a user may define an expected level for one or more metrics of QoE that the user expects in a service contract (e.g., a service level agreement (SLA)) with a service provider. The expected level of one or more metrics (referred to herein as "SLA metrics") is typically a user-configurable value and is derived through trial and error or a benchmarking environment with user experience or real-world best application metrics. The network device 13 of the SD-WANss 7 (e.g., the SD-WAN device 18) can determine the best path for forwarding traffic flows based on the SLA metrics. However, the user-configured SLA metric values ​​are laborious and subjective to the benchmark environment suggested by the provider. For example, a network system may typically include a large number of applications, each of which requires the user to configure the SLA metric to achieve a suggested benchmark. In addition, any changes to the suggested benchmark will require the user to manually configure each SLA metric.

[0036] According to the techniques described herein, the network system 2 performs dynamic application SLA metric generation, distribution, and intent-based SD-WAN link selection. For example, the network system 2 may include an application SLA metric controller 14 ("SLA controller 14") that receives one or more metric values ​​for each application and / or application group from each network device 13 of the SD-WAN 7 managed by the SLA controller 14 (referred to as an "SD-WAN leaf node" of the SLA controller 14), and distributes recommended SLA metrics for each application and / or application group to the SD-WAN leaf node, so that the SD-WAN leaf node can (partially) calculate the intent-based SLA metric for each application and / or application group, wherein the intent-based SLA metric is used to determine the optimal / best path or link for sending traffic for the corresponding application and / or application group.

[0037] exist Figure 1 In the example shown, each SD-WAN leaf node can dynamically learn one or more metrics for each application and / or application group in real time or near real time. For example, the SD-WAN leaf node can use TWAMP (or other measurement protocols) to exchange test packets on its links to measure end-to-end metrics for each application and / or application group. Using the test packets, each SD-WAN leaf node can calculate metrics such as path connectivity, path delay, packet jitter, packet loss, packet reordering, etc. The SD-WAN leaf nodes can each store the learned metrics for each application in a data structure (such as a database). Figure 2 As further described in, the SD-WAN leaf node can also associate each application with an application group and store this information in a data structure.

[0038] The SLA controller 14 may obtain one or more learned metrics from each SD-WAN leaf node. For example, each SD-WAN leaf node may send one or more learned metrics for a specific application and / or application group to the SLA controller 14. For example, each SD-WAN leaf node may send one or more learned metrics for a specific application. Alternatively or additionally, each SD-WAN leaf node may send one or more learned metrics for applications in a group (e.g., web applications, or non-real-time and asymmetric (NRTA) applications). Each SD-WAN leaf node may send one or more learned metrics in batches, or in some cases when they are learned. In some cases, the SD-WAN leaf node may periodically send one or more metrics based on a configured interval (e.g., every 15 minutes). Each SD-WAN leaf node may send one or more learned metrics using a secure socket layer (SSL), transport layer security (TLS), hypertext transport layer (HTTP), secure hypertext transfer protocol (HTTPS), or other protocols to establish a communication channel (and in some cases a secure communication channel) between the SLA controller 14 and the SD-WAN leaf node. In some examples, the SLA controller 14 may pull one or more learned metrics from each SD-WAN leaf node, such as by initiating a request to receive one or more learned metrics from the SD-WAN leaf node or directly retrieving one or more learned metrics from a data structure of each SD-WAN leaf node.

[0039] The SLA controller 14 receives one or more learned metrics from the SD-WAN leaf nodes and can calculate recommended SLA metrics, i.e., ideal user experience, for each application and / or application group. For example, the SLA controller 14 can determine the aggregated mean (and in some cases the standard deviation) of the metrics based on the aggregation of the metrics of the corresponding applications and / or application groups obtained from the SD-WAN leaf nodes. The SLA controller 14 can calculate the recommended SLA metrics for each application and / or application group. The SLA controller 14 pushes the recommended SLA metrics for each application and / or application group to each SD-WAN leaf node. For example, the SLA controller 14 can send the recommended SLA metrics using a secure communication channel (e.g., SSL) or as plain text.

[0040] When the SD-WAN leaf node receives the recommended SLA metrics from the SLA controller 14, the SD-WAN leaf node may perform intent-based SLA metric calculation and link selection for each application and / or application group. Figure 3As further described in, each SD-WAN leaf node can calculate intent-based SLA metrics based on recommended SLA metrics received from the SLA controller 14, one or more characteristics of links connected to the SD-WAN leaf node, and a user-configured intent model that defines desired tolerance levels to further fine-tune the recommended SLA metrics received from the SLA controller 14.

[0041] Each SD-WAN leaf node may calculate an end-to-end best constrained path metric for each application and / or application group based on the intent-based SLA metric. For example, each SD-WAN leaf node may use a shortest path algorithm to determine the optimal / best path for forwarding application traffic, such as Dijkstra's algorithm, Bellman-Ford's algorithm, Breadth First Search (BFS), Depth First Search (DFS), Floyd's algorithm, Johnson's algorithm, Kruskal's algorithm, Primm's algorithm, Algorithm or similar algorithm. See below for reference Figure 4 As further described, when using a shortest path algorithm, each SD-WAN leaf node can determine a path based on per-hop metrics for each application and / or each application group (e.g., the link that best meets the intent-based SLA metric).

[0042] By applying a shortest path algorithm, each SD-WAN leaf node may have end-to-end visibility of links that meet and / or do not meet intent-based SLA metrics per application and / or application group. For example, an SD-WAN leaf node may implement a shortest path algorithm and may determine which links meet or do not meet intent-based SLA metrics. The SD-WAN leaf node may store the determined paths and information indicating which links meet and / or do not meet intent-based SLA metrics.

[0043] In some examples, each SD-WAN leaf node can monitor network performance over a period of time. For example, each SD-WAN leaf node can feed one or more learned metrics into a machine learning model (which can be executed on the leaf node in some examples), such as a support vector machine (SVM) model, a K nearest neighbor (KNN) model, a logistic regression model, and / or a deep learning model. The SD-WAN leaf node can use the machine learning model to classify the observed metrics and learn accurate values ​​over a period of time. In one implementation, the machine learning model of the SD-WAN leaf node can calculate a mean opinion score, as described in the Mean Opinion Score (MOS) terminology of the International Telecommunication Union's Telecommunication Standardization Sector (ITU-T) P.800.1, entitled "SERIES P:TERMINALS AND SUBJECTIVE AND OBJECTIVEASSESSMENT METHODS, Methods for objective and subjective assessment of speechand video quality" in July 2016, the entire contents of which are incorporated herein by reference. MOS can represent a single rational number (e.g., 1-5), where 1 is the lowest perceived quality and 5 is the highest perceived quality. The machine learning model of the SD-WAN leaf node can calculate the estimated MOS (EMOS) for application groups to evaluate the link behavior of any application belonging to these groups. The machine learning model of the SD-WAN leaf node can perform a permanent evaluation of link performance metrics and derive estimated MOS for different application categories. That is, the link is continuously monitored and the performance metrics of latency, jitter, packet loss, etc. are correlated to obtain link MOS scores for different application categories. The estimated MOS is then converted into an SLA target metric and the SD-WAN leaf node is dynamically updated.

[0044] exist Figure 1 In the example shown, the SD-WAN device 18 (e.g., an SD-WAN leaf node) can determine the shortest path to the router 8 based on a per-hop metric (e.g., an intent-based SLA metric) for each application and / or application group. The SD-WAN device 18 can select a path along one or more of the interconnected communication links that meets the intent-based SLA metric calculated by the SD-WAN device 18. Each network device 13 of the SD-WAN 7 can also select a path along one or more of the interconnected communication links that meets the intent-based SLA metric calculated by the network device. Each network device 13 of the SD-WAN 17 can also store the network health of the link (e.g., whether the link meets the intent-based SLA metric).

[0045] In some examples, each SD-WAN leaf node can determine the shortest path when a user-configured intent model is changed (e.g., from an aggressive intent model to a fragile intent model). For example, the SD-WAN leaf node can be configured with an aggressive intent model for an application at an initial instance (e.g., T1). For example, when the SD-WAN receives traffic from an application, the SD-WAN leaf node can select a link from a plurality of links that meets one or more SLA metrics within a tolerance threshold (e.g., a tolerance of 10-15% for a recommended SLA metric). In a subsequent instance (e.g., ΔT), the user can change the aggressive intent model to a conservative intent model for the application. When the SD-WAN leaf node receives traffic from an application, the SD-WAN leaf node can select a link from a plurality of links that applies a more stringent metric data set relative to one or more recommended SLA metrics.

[0046] The technology described herein can provide one or more technical advantages that can provide at least one practical application. For example, the technology described herein provides dynamic application and / or application group metric learning from real-world network deployments through all connected SD-WAN leaf nodes. Each SD-WAN leaf node applies the SLA metric of the best user experience from real customer application services (rather than from laboratory design / test environment application services). By periodically sending the recommended SLA metric for each application and / or application group by the SLA controller, the administrator can be exempted from the heavy metric configuration work for each leaf node and application when the WAN link capacity changes. The automatic SLA metric calculation at the SLA controller can release the extensive product engineering work of the SD-WAN supplier, that is, each application and / or application group statically calculates the metric for recommendation to the customer. In addition, the SLA controller continuously learns and calculates the metric, and recommends the SLA metric that can be used for each application and / or application group. By dynamically learning and calculating the metric and recommending the SLA metric for each application and / or application group, the calculation of the SLA metric can provide more accurate and up-to-date SLA metrics relative to the manually configured SLA metric. The intent-based expected SLA metric calculation at the SD-WAN leaf node is adapted to the network interface conditions and the user-configured intent model on the corresponding node, which enables the user to further fine-tune the recommended SLA metrics from the SLA controller. In addition, one or more aspects of the described technology provide full network visibility about which links meet or do not meet the expected SLA metrics in the SD-WAN network. This enables the discovery of paths / links in the SD-WAN that do not meet SLA requirements, which can be used to provide instructions to resolve the problem (e.g., by upgrading the link).

[0047] Figure 2is a block diagram illustrating example dynamic application SLA metric generation and distribution of recommended SLA metrics for an SD-WAN in accordance with one or more aspects of the techniques described in this disclosure.

[0048] exist Figure 2 In the example shown, the SD-WAN leaf nodes 204A-204N (collectively referred to as "SD-WAN leaf nodes 204") may represent Figure 1 Any network device 13 of the SD-WAN 7. Each SD-WAN leaf node 204 can dynamically learn one or more metrics for each application and / or application group in real time or near real time. For example, the SD-WAN leaf nodes 204 can each use TWAMP (or other measurement protocols) to exchange test packets to measure end-to-end metrics for each application and / or application group, and store the learned metrics for each application in a data structure, such as a corresponding one of the databases 206A-206N (collectively referred to as "databases 206").

[0049] In this example, the SD-WAN leaf node 204A can learn the metrics RTT1, JITTER1, and / or LATENCY1 of the first application APP1 and store this information in the database 206A. Similarly, the SD-WAN leaf node 204A can learn one or more metrics of other applications (e.g., APP2-APP4). Each of the SD-WAN leaf nodes 204B-204N can also learn one or more metrics for each application.

[0050] Each SD-WAN leaf node 204 may also associate each application with an application group. For example, each application may be associated with an application group based on a corresponding quality of service standard (otherwise known as a service level) ( Figure 2 One or more applications may be grouped together as applications with a gold standard. One or more applications may require a medium priority and / or medium dedicated bandwidth and may be grouped together as applications with a silver standard. One or more applications may require a lower priority or lower dedicated bandwidth and may be grouped together as applications with a copper standard. Although described with respect to gold, silver, and copper QoS standards, one or more applications may be grouped according to other QoS standards, such as platinum, which groups applications requiring the highest priority or the highest dedicated bandwidth (e.g., voice applications such as VoIP).

[0051] Applications may also be grouped in categories based on the type of application, such as web, financial, educational, social networking, avoidance applications, or other types of applications. For example, the SD-WAN leaf node 204A may associate APP1 and other social applications with the application group SOCIAL, APP2 and other web applications with the application group WEB, etc. Each of the SD-WAN leaf nodes 204B-204N may also associate one or more applications with an application group.

[0052] Alternatively or additionally, one or more applications may be classified based on one or more characteristics of the application, such as whether the application is non-real-time and asymmetric (NRTA), non-real-time and symmetric (e.g., NRTS), real-time and asymmetric (RTA), and real-time and symmetric (RTS). NRTA applications generate services that are considered best-effort services. That is, the services are transmitted using a best-effort protocol. The best-effort protocol does not provide any specific QoS reliability, but simply processes services on a first-come, first-served basis. Common NRTA applications include web browsing—Hypertext Transfer Protocol (HTTP), email, file transfer protocol (FTP), and remote login. NRTS applications typically involve client devices that request data and services from a host device and require equivalent resource consumption at each device. Examples of NRTS applications include Internet relay chat. Due to the nature of real-time transmission, RTA applications have QoS requirements, which are usually strict QoS requirements. Common RTA applications include audio broadcasting, video broadcasting, on-demand interactive audio, on-demand interactive video, and telemetry. RTS applications may include applications that are essentially conversational, such as teleconferencing (including audio, audio, and video conferencing), video telephony, and voice over IP (VoIP).

[0053] exist Figure 2 In the example shown, the SD-WAN leaf node 204A can associate applications APP1-APP3 with the application category of NRTA, and associate application APP4 with the application category of RTS. Although the examples described herein are described for specific groups based on QoS criteria or application types, the techniques described herein can be applied to any type of grouping of multiple applications. Application groups can be predefined and / or user-defined.

[0054] The SLA controller 14 may obtain one or more learned metrics from each SD-WAN leaf node 204. For example, each SD-WAN leaf node 204 may send one or more learned metrics for a particular application and / or group of applications stored in its database to the SLA controller 14. In some examples, the SLA controller 14 may pull the one or more learned metrics from the database of each SD-WAN leaf node 204, such as by initiating a request to receive the one or more learned SLA metrics from the SD-WAN leaf node 204 or accessing the database of each SD-WAN leaf node 204 to retrieve the one or more learned metrics stored in the database.

[0055] exist Figure 2 In the example shown, the SD-WAN leaf node 204A may send metrics including RTT1, JITTER1, and LATENCY1 of the first application APP1 to the SLA controller 14 using HTTP. For example, the SD-WAN leaf node 204A may send an HTTP message 208A with metrics for APP1. Similarly, the SD-WAN leaf node 204B may send an HTTP message 208B with metrics including RTT1, JITTER1, and LATENCY1 of the first application APP1. Likewise, the SD-WAN leaf node 204N may send an HTTP message 208N with metrics including RTT1, JITTER1, and LATENCY1 of the first application APP1. Although Figure 2 The example shown uses HTTP to send the SLA metrics, the SD-WAN leaf nodes may implement any communication protocol to establish a communication session with the SLA controller 14.

[0056] Alternatively or additionally, each SD-WAN leaf node 204 may send one or more learned metrics of an application group. For example, the SD-WAN leaf node 204A may send one or more learned metrics of each application associated with the application category NRTA. In this example, the SD-WAN leaf node 204A may send one or more messages with one or more metrics of applications associated with the application category NRTA. For example, the SD-WAN leaf node 204A may send a first message with RTT1, JITTER1, and LATENCY1 of the first application APP1, a second message with RTT2, JITTER2, and LATENCY2 of the second application APP2, and a third message with RTT3, JITTER3, and LATENCY3 of the third application APP3. In some examples, the SD-WAN leaf node 204A may send metrics of applications APP1-APP3 in a single message. Similarly, each of the SD-WAN leaf nodes 204B-204N may also send one or more messages with one or more metrics of applications associated with the application category NRTA.

[0057] Each SD-WAN leaf node 204 may send one or more learned metrics in batches, or in some cases as they are learned. In some cases, the SD-WAN leaf node may send one or more metrics periodically based on a configured interval (e.g., every 15 minutes).

[0058] In response to receiving one or more learned metrics from each SD-WAN leaf node 204, the SLA controller 14 may calculate a recommended SLA metric, i.e., an ideal user experience, for each application and / or application group. For example, the SLA controller 14 may include a recommended SLA metric module 212 that may calculate an aggregated mean (and in some cases, a standard deviation) of the metrics for each application and / or application group based on an aggregation of the metrics obtained from each SD-WAN leaf node 204. In this example, the recommended SLA metric module 212 may determine the average RTT (in the example) based on an aggregation of the RTT1 values ​​for APP1 received from each SD-WAN leaf node 204. Figure 2 Additionally or alternatively, the recommended SLA metric module 212 may determine the average RTT ( RTT_A) based on an aggregation of the RTT values ​​of APP1-APP3 associated with the application group NRTA received from each SD-WAN leaf node 204. Figure 2 Shown as 214B).

[0059] The SLA controller 14 pushes the recommended SLA metrics for each application and / or application group to each SD-WAN leaf node. For example, the SLA controller 14 may send the recommended SLA metrics included in each message 210A-210N (collectively referred to as "recommended SLA metrics message 210") to the SD-WAN leaf node 204. The recommended SLA metrics message 210 may be sent using a secure communication channel (e.g., SSL) or as plain text. Figure 2 In the example of , the SLA controller 14 may send to the SD-WAN leaf node 204 a recommended SLA metric (e.g., RTT) for an application (e.g., APP1) and / or application group (e.g., NRTA) to be used in part by the SD-WAN leaf node 204 to perform intent-based link selection, such as Figure 3 Further described in .

[0060] Figure 3 is a block diagram illustrating an example of logic for intent-based SLA metric calculations in accordance with one or more aspects of the techniques described herein.

[0061] When the SD-WAN leaf node 204 receives the recommended SLA metrics for each application and / or application group from the SLA controller 14, the SD-WAN leaf node may perform an intent-based SLA metric calculation 300 for each application and / or application group. That is, the SD-WAN leaf node 204 applies the intent-based model to the received recommended SLA metrics, which may, in some cases, result in modifications to the recommended SLA metrics. Figure 3In the example shown, the SD-WAN leaf node 204A can calculate an intent-based SLA metric 310 based on a recommended SLA metric 304 received from the SLA controller 14, one or more SD-WAN link characteristics 306, and a user-configured intent model 308. As described above, the recommended SLA metric 304 is a mean of a metric (e.g., RTT) for an application or group of applications received from the SLA controller 14. The SD-WAN link characteristics 306 may include different characteristics of different types of links connected to the SD-WAN leaf node. For example, the links may include physical links (such as 100 Gigabit Ethernet links and / or satellite links) and wireless links (such as wireless links configured with third generation mobile telecommunication (3G) standards, fourth generation mobile telecommunication (4G), fifth generation mobile telecommunication (5G) standards, or other wireless technologies). The characteristics of the link depend on the type of link. For example, if the SD-WAN leaf node is connected to a physical link, the link characteristics 306 may include characteristics for the following: link type (e.g., media fiber or copper cable), whether data rates are supported, whether the SD-WAN leaf node is a transceiver, whether the provider specifies the characteristics, signal strength, and other characteristics. In some examples, if the SD-WAN leaf node is connected to a wireless link, the link characteristics 306 may include characteristics such as latency, signal strength, coverage, data rate, bandwidth per unit area, frequency, provider, and other characteristics.

[0062] The user-configured intent model 308 may include a model that defines a desired tolerance level for further fine-tuning the recommended SLA metrics received from the SLA controller 14. For example, a user may configure an aggressive intent model that causes the SD-WAN leaf node to aggressively apply the recommended SLA metrics. In some examples, the tolerance level of the aggressive intent model is configured to be a 10%-15% tolerance for the recommended SLA metrics. If the metric of a link falls within 10%-15% of the recommended SLA metric, the SD-WAN leaf node may select the link. The user may alternatively or additionally configure a normal intent model that causes the SD-WAN leaf node to apply the recommended SLA metrics as received without modification. Alternatively or additionally, the user may configure a fragile intent model that causes the SD-WAN leaf node to conservatively apply the recommended SLA metrics. For example, the SD-WAN leaf node may select a link that applies a more stringent set of metrics relative to the recommended SLA metrics. As Figure 4 As further described in, each SD-WAN leaf node 204 can calculate an end-to-end optimal constrained path metric for each application and / or application group based on the intent-based SLA metric.

[0063] Figure 4is a block diagram illustrating an example technique for path selection based on intent-based SLA metric calculation for SD-WAN in accordance with one or more aspects of the techniques described herein.

[0064] When the SD-WAN leaf nodes receive the recommended SLA metrics 404A-404N (collectively, "recommended SLA metrics 404") from the SLA controller 14, the SD-WAN leaf nodes 204A-204G (collectively, "SD-WAN leaf nodes 204") may perform intent-based SLA metric calculation and link selection for each application and / or application group. As described above, each SD-WAN leaf node 204 may calculate an end-to-end optimal constrained path metric for each application and / or application group based on the intent-based SLA metrics derived from the recommended SLA metrics 404, one or more SD-WAN link characteristics of the links connected to the router, and a user-configured intent model defining the desired tolerance level to further fine-tune the recommended SLA metrics received from the SLA controller 14. Each SD-WAN leaf node 204 may then apply a shortest path algorithm using the intent-based SLA metrics per hop to determine the optimal / best path for sending traffic for the application and / or application group.

[0065] exist Figure 4 In the example shown, each of the SD-WAN leaf nodes 204A-204G is connected by links 412A-412L (collectively referred to as "links 412"). Each of the SD-WAN leaf nodes 204A-204G can be based on the recommended SLA metric 404A for the first application or the first application group, the characteristics of the connecting link, and the above Figure 3 412B and 412C, and the intent model configured by the user to calculate the intent-based SLA metric for the first application or the first application group. For example, the SD-WAN leaf node 204A can calculate the intent-based SLA metric for the first application based on the recommended SLA metric 404A, the link characteristics of the links 412A, 412B, and the user-configured intent model. The SD-WAN leaf node 204B can calculate the intent-based SLA metric for the first application or the first application group based on the recommended SLA metric 404A, the link characteristics of the links 412C, 412D, and 412E, and the user-configured intent model. The other SD-WAN leaf nodes 204C-204G can each calculate the intent-based SLA metric for the first application or the first application group.

[0066] In response to applying the shortest path algorithm according to the intent-based SLA metric, the SD-WAN leaf node 204 may calculate a path 410A for sending traffic of the first application or the first application group through the SD-WAN leaf nodes 204A, 204B, 204F, and 204G. For example, the SD-WAN leaf node 204A may determine that the link 412B satisfies the intent-based SLA metric of the first application or the first application group, the SD-WAN leaf node 204B may determine that the link 412E satisfies the intent-based SLA metric of the first application or the first application group, and the SD-WAN leaf node 204F may determine that the link 412G satisfies the intent-based SLA metric of the first application or the first application group. In this way, the SD-WAN leaf nodes 204A, 204B, 204F, and 204G are each configured to forward traffic of the first application or the first application group along the path 410A.

[0067] As another example, each of the SD-WAN leaf nodes 204A-204G may calculate an intent-based SLA metric for the second application or second application group based on the recommended SLA metric 404B for the second application or second application group, characteristics of the connecting link, and the user-configured intent model. In response to applying a shortest path algorithm based on the intent-based SLA metric, the SD-WAN leaf node 204 may calculate a path 410B for sending traffic for the second application or second application group through the SD-WAN leaf nodes 204A, 204B, 204D, and 204G. For example, the SD-WAN leaf node 204A may determine that link 412B satisfies the intent-based SLA metric for the second application or second application group, the SD-WAN leaf node 204B may determine that link 412D satisfies the intent-based SLA metric for the second application or second application group, and the SD-WAN leaf node 204D may determine that link 412L satisfies the intent-based SLA metric for the second application or second application group. In this way, the SD-WAN leaf nodes 204A, 204B, 204D and 204G are all configured to forward the traffic of the second application or the second application group along the path 410B.

[0068] As yet another example, each of the SD-WAN leaf nodes 204A-204G may calculate an intent-based SLA metric for the Nth application or the Nth application group based on the recommended SLA metric 404N for the Nth application or the Nth application group, characteristics of the connecting link, and the user-configured intent model. In response to applying a shortest path algorithm based on the intent-based SLA metric, the SD-WAN leaf node 204 may calculate a path 410N for sending traffic for the Nth application or the Nth application group through the SD-WAN leaf nodes 204A, 204C, 204E, and 204G. In this way, the SD-WAN leaf nodes 204A, 204C, 204E, and 204G are each configured to forward traffic for the Nth application or the Nth application group along the path 410N.

[0069] Figure 5 is a block diagram illustrating an example SLA controller configured in accordance with one or more aspects of the techniques described herein. Figure 5 In the example of Figure 1-Figure 4 Example of an SLA controller 14. Figure 5 As shown in the example of , SLA controller 500 can represent a collection of tools, systems, devices, engines, and modules that perform operations according to one or more aspects of the technology described in this disclosure. Figure 5 The SLA controller 500 can be combined with Figure 1-Figure 4 The SLA controller 14 is implemented in a manner consistent with the description provided.

[0070] exist Figure 5 In the example of , the SLA controller 500 includes SLA metrics 502. The SLA metrics 502 can store one or more metrics received from the SD-WAN leaf nodes 204A-204N. In some examples, the SLA metrics 502 can be implemented as any type of data structure, such as a data store or database. In such an example, the SLA metrics 502 can represent any suitable data structure or storage medium for storing metrics and / or information related to metrics received from the SD-WAN leaf nodes 202.

[0071] exist Figure 5In the example of , the SLA controller 500 includes a device manager 504 that can communicate with one or more SD-WAN leaf nodes 204 to obtain one or more metrics for each application and / or application group and / or send recommended SLA metrics for each application and / or application group to each SD-WAN leaf node 204. The device manager 504 can implement various communication protocols / techniques, such as a secure socket layer 506 (“SSL 506”), to receive one or more metrics from each SD-WAN leaf node 204 or send recommended SLA metrics to each SD-WAN leaf node 204. Although the device manager 504 is shown as implementing SSL 506, the device manager 504 can implement additional or alternative protocols, such as transport layer security (TLS), hypertext transfer protocol (HTTP), secure hypertext transfer protocol (HTTPS), to establish a communication channel (in some cases a secure communication channel) between the SLA controller 500 and the SD-WAN leaf node 204. The manager 504 can store the one or more metrics received from the SD-WAN leaf node 204 in the SLA metrics 502. The device manager 504 can retrieve one or more recommended SLA metrics for each application and / or application group stored in the SLA metrics 502 and send the one or more recommended SLA metrics to the SD-WAN leaf nodes 204 to enable each SD-WAN leaf node 204 to calculate an intent-based SLA metric for determining a path for sending traffic for the application or application group.

[0072] The SLA controller 500 may include a recommended SLA metrics module 508 for calculating recommended SLA metrics based on an aggregation of metrics for applications and / or application groups received from each SD-WAN leaf node 204. The recommended SLA metrics module 508 may be used in conjunction with Figure 2 The recommended SLA metric module 508 may be implemented in a manner consistent with the description of the SLA metric module 212 provided above. For example, the recommended SLA metric module 508 may retrieve an aggregation of corresponding SLA metrics for corresponding applications and / or application groups from the SLA metrics 502 and calculate a mean (or standard deviation in some cases) of the aggregation of corresponding metrics. The recommended SLA metric module 508 may store the recommended SLA metric in the SLA metrics 502.

[0073] The device manager 504 may retrieve recommended SLA metrics for applications and / or application groups from the SLA metrics 502 and send the recommended SLA metrics to the SD-WAN leaf node 204 using, for example, SSL 506.

[0074] Figure 6 is a block diagram illustrating an example network device configured in accordance with one or more aspects of the techniques described herein. Figure 6The network device 600 may be further represented in detail Figure 1 Network device 13 and Figure 2-Figure 5 SD-WAN leaf node 204.

[0075] The network device 600 includes a control unit 602, which includes a routing engine 604 and is coupled to a forwarding engine 606 (also referred to herein as "forwarding unit 606"). The forwarding engine 606 is associated with one or more of the interface cards 632A-632N ("IFCs 632") that receive packets via inbound links 658A-658N ("inbound links 658") and send packets via outbound links 660A-660N ("outbound links 660"). The IFCs 632 are typically coupled to the links 658, 660 via a plurality of interface ports (not shown). The interfaces of the inbound links 658 and the outbound links 660 may represent physical interfaces, logical interfaces, or some combination thereof. The interfaces of the links 658, 660 may represent interfaces for receiving packets from the network. Figure 1 The WAN link of the SD-WAN 7 network device is the local interface of the network device 600.

[0076] In general, control unit 602 may represent hardware or a combination of hardware and control software that implements one or more protocols 620 to learn and maintain routing information 634. Routing information 634 may include definitions such as Figure 1 The routing information may define the topology of a network such as a service provider network 2. For example, the routing information may define a route (e.g., a series of next hops) through the network to a destination / prefix within the network learned via a distance vector routing protocol (e.g., BGP 624), or define a network topology with interconnected links learned using a link state routing protocol (e.g., ISIS or OSPF) using IGP 622. The protocol 620 is associated with a kernel ( Figure 6 The control unit 602 may interact with the microprocessor 614 (e.g., via API calls) to update the routing information 634 based on the routing protocol messages received by the network device 600. The kernel of the control unit 602 executes the main microprocessor 614 and may include, for example, a UNIX operating system derivative such as Linux or Berkeley Software Distribution (BSD). The kernel provides libraries and drivers that user-level processes can use to interact with the underlying system. The microprocessor 614 executes the program from the storage device ( Figure 6 ) into main memory (not shown) Figure 6The microprocessor 614 may represent one or more general or special purpose processors, such as a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or any other equivalent logic device. Therefore, the term "processor" or "controller" as used herein may refer to any one or more of the aforementioned structures or any other structures operable to perform the techniques described herein.

[0077] The kernel processes kernel calls from the routing protocol 620 to generate forwarding information 608 based on the network topology represented in the routing information 634. Typically, the forwarding information 608 is generated in the form of a radix or other lookup tree to map packet information (e.g., header information with destination information and / or label stack) to a next hop and ultimately to an interface port of an IFC 632 associated with the forwarding engine 606. The forwarding information 608 can associate, for example, a network destination with a specific next hop and a corresponding IFC 632. For traffic forwarding related to MPLS, the forwarding information 808 stores label information and the next hop of the packet, the label information including an incoming label and an outgoing label. The control unit 602 can then use the forwarding information 608 to program the forwarding engine 606 of the data plane of the network device, for example, in an application specific integrated circuit (ASIC) ( Figure 6 Forwarding information is installed in (not shown).

[0078] Figure 6The architecture of the network device 600 shown is shown for example purposes only. The present disclosure is not limited to the architecture. In other examples, the network device 600 can be configured in various ways. In one example, some functions of the control unit 602 can be distributed in the IFC 632. The control unit 602 can be implemented only in software or hardware, or can be implemented as a combination of software, hardware or firmware. For example, the control unit 602 may include a processor, a programmable processor, a general-purpose processor, a digital signal processor (DSP), an integrated circuit, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or one or more of any type of hardware units capable of implementing the technology described herein. The control unit 602 may also include one or more processors that execute software instructions stored, implemented or encoded in a computer-readable medium (such as a computer-readable storage medium). The instructions embedded or encoded in the computer-readable medium can enable a programmable processor or other processor to perform the method, for example, when the instructions are executed. Computer-readable storage media may include random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), non-volatile random access memory (NVRAM), flash memory, hard disk, CD-ROM, floppy disk, cassette, solid-state drive, magnetic media, optical media, or other computer-readable media. In some cases, the computer-readable storage medium may include instructions that cause a programmable processor to perform the techniques described herein.

[0079] exist Figure 6 In the example of , routing engine 604 provides an operating environment for one or more traffic engineering protocols to establish tunnels for forwarding subscriber packets through ordered service node sets 10 associated with different service chains. For example, resource reservation protocol with traffic engineering extension (RSVP-TE) 626 can exchange traffic engineering information, such as MPLS labels for enabling label-based packet forwarding. As another example, routing engine 604 can use a tunneling protocol (not shown) based on GRE or IP to establish a traffic engineering tunnel. Routing engine 604 can maintain, for example, a traffic engineering database (TED) ( Figure 6 Although not shown in Figure 6 , but protocol 620 may include other protocols, such as label distribution protocol (LDP) or other traffic engineering protocols.

[0080] Routing engine 604 provides an operating environment for various application quality of experience (QoE) measurement functions, such as real-time performance monitoring (RPM), one-way active measurement protocol (OWAMP), two-way active measurement protocol (TWAMP), active detection, passive detection, protocol deep packet inspection (DPI) engine, or other measurement protocols or technologies. Figure 6 In the example shown, routing engine 604 can use TWAMP 628 to perform one or more TWAMP logical roles, such as TWAMP control client, TWAMP server, TWAMP session transmitter, and TWAMP session reflector. The execution of one or more TWAMP logical roles enables network device 600 to exchange test packets for measuring one or more metrics.

[0081] The routing engine 604 provides an operating environment for the SSL 629. For example, the routing engine 604 may use the SSL 629 to establish a secure communication channel with the SLA controller to send measured metrics to the SLA controller and receive recommended SLA metrics from the SLA controller. The protocol 620 may include Figure 6 Other communication protocols not shown in the figure (eg, TLS, HTTP) are used to provide a communication channel with the SLA controller.

[0082] According to the techniques described in the present invention, routing engine 604 may include an SLA module 640 that can dynamically learn one or more metrics in real time or near real time for each application and / or application group. As described above, routing engine 604 can implement TWAMP 628 to exchange test packets with another network device to perform link state measurements. SLA metric module 640 can determine one or more metrics for each application and / or application group based on the test packets. SLA metric module 640 can store the learned one or more metrics in SLA metrics 646.

[0083] The SLA metrics 646 may store one or more learned metrics for each application and / or application group. The SLA metrics 646 may store one or more metrics, applications associated with one or more metrics, application groups associated with applications, or other information associated with one or more metrics. In some examples, the SLA metrics 646 may be implemented as any type of data structure, such as a data store or a database. In such examples, the SLA metrics 646 may represent any suitable data structure or storage medium for storing metrics and / or information related to metrics dynamically learned by the network device 600.

[0084] Routing engine 604 can implement SSL 629 to send one or more learned metrics for each application and / or application group to the SLA controller. Network device 600 can also receive one or more recommended SLA metrics for each application and / or application group from the SLA controller using SSL 629. In such an example, network device 600 can send an SSL message with the one or more learned metrics and receive an SSL message with the one or more recommended SLA metrics from the SLA controller.

[0085] The SLA metrics module 640 may include an intent module 642 that may calculate intent-based SLA metrics for each application and / or application category. For example, the intent module 642 may calculate intent-based SLA metrics for an application or application group based on recommended SLA metrics for the application or application category received from the SLA controller, characteristics of one or more links connected to the network device 600 (e.g., links connected to one or more IFCs 632), and a user-configured intent model for the application or application group.

[0086] As described above, the user-configured intent model defines the desired tolerance levels for further fine-tuning the recommended SLA metrics received from the SLA controller. Figure 6 In the example shown, a user can configure the intent model using a configuration interface 650 of routing engine 604. In some examples, configuration interface 650 can represent a command line interface, a graphical user interface, a simple network management protocol (SNMP), NETCONF or other configuration protocol, or some combination thereof. Configuration interface 650 receives configuration data that configures one or more user-configured intent models for fine-tuning recommended SLA metrics received from a SLA controller.

[0087] The SLA metrics module 640 may include an SLA path module 644 for establishing a path between endpoints on a path computation domain. Paths may be requested based on various SLA metrics. That is, the SLA path module 644 may select a path for a corresponding application and / or application group that satisfies a given SLA metric. In order to compute a path through the path computation domain, the SLA path module 644 may discover the topology of the network (e.g., by executing IGP 622 or BGP 624 to receive routing protocol announcements), and may receive a list of node neighbors (where each neighbor includes a node identifier, a local port index, and a remote port index) and a list of link attributes (where each link attribute specifies a port index, a bandwidth, an expected transmission time, a shared link group, and a fate sharing group).

[0088] The SLA path module 644 can select a path along one or more interconnected communication links that satisfies the intent-based SLA metric calculated by the intent module 642. For example, the SLA path module 644 can implement one or more shortest path algorithms, as described above. In response to implementing the shortest path algorithm, the SLA path module 644 can determine whether each interconnected communication link satisfies the intent-based SLA metric. The SLA path module 644 can store the determined path and an indication of each interconnected communication link as to whether the link satisfies the intent-based SLA metric.

[0089] Figure 7 is a flow chart illustrating an example technique for an SD-WAN to perform dynamic application service level agreement metric generation, distribution, and intent-based SD-WAN link selection in accordance with one or more aspects of the techniques described in this disclosure. Figure 5 The SLA Controller 500 and Figure 6 The following are the steps of the process, but other examples of processes performed in the technology of the present disclosure may include additional steps, or may not include some of the steps listed below.

[0090] According to the techniques described herein, an SD-WAN leaf node in a plurality of SD-WAN leaf nodes may learn one or more metrics per application and / or application group (702). For example, each SD-WAN leaf node may use various application quality of experience (QoE) metric functions such as RPM, OWAMP, TWAMP, active probing, passive probing, protocol DPI engine, or other measurement protocols or techniques for learning one or more metrics per application and / or application group for its links. The SD-WAN leaf nodes may each store the learned one or more metrics in a local SLA metrics data structure.

[0091] Each SD-WAN leaf node may send one or more metrics for each application and / or application group learned to the SLA controller (704). For example, each SD-WAN leaf node may use a protocol to establish a communication channel, such as a secure socket layer (SSL), a transport layer security (TLS), a hypertext transfer layer (HTTP), or a secure hypertext transfer protocol (HTTPS). Each SD-WAN leaf node may send one or more learned metrics stored in its SLA metric data structure to the SLA controller. For example, each SD-WAN leaf node may send one or more learned metrics for a specific application to the SLA controller. Alternatively or in addition, each SD-WAN leaf node may send one or more learned metrics for a group of applications (e.g., web applications or NRTA applications) to the SLA controller. Each SD-WAN leaf node may send one or more learned metrics in batches, or in some cases when they are learned. In some cases, the SD-WAN leaf node may send one or more metrics periodically based on a configured interval (e.g., every 15 minutes).

[0092] The SLA controller may receive one or more metrics (706) from each SD-WAN leaf node and calculate a recommended SLA metric (708) for each application and / or application group. For example, the SLA controller may receive metrics for an application or application group from each SD-WAN leaf node. The SLA controller may include a recommended SLA metric module 508 for calculating an aggregated mean (and in some cases a standard deviation) of the metrics for the applications and / or application groups. The SLA controller may calculate an aggregated mean of the SLA metrics for each application and / or application group.

[0093] The SLA controller may send the recommended SLA metrics for each application and / or application group to each SD-WAN leaf node (710). For example, the SLA controller may send the recommended SLA metrics using SSL.

[0094] When the SD-WAN leaf node in the plurality of SD-WAN leaf nodes receives the recommended SLA metrics for each application and / or application group (712), the SD-WAN leaf node may calculate the intent-based SLA metrics (714). For example, the SD-WAN leaf node may include an intent module 642 of the SLA metric module 640 for calculating the intent-based SLA metrics based on the recommended SLA metrics received from the SLA controller, one or more characteristics of the links connected to the SD-WAN leaf node, and a user-configured intent model that defines a desired tolerance level to further fine-tune the recommended SLA metrics received from the SLA controller. For example, a user may configure (e.g., via the configuration interface 650) an aggressive intent model that may cause the SD-WAN leaf node to aggressively apply the recommended SLA metrics (e.g., select a link with a better RTT than the recommended SLA metrics). Alternatively or additionally, a user may configure a normal intent model that may cause the SD-WAN leaf node to apply the recommended SLA metrics as is. Alternatively or additionally, a user may configure a fragile intent model that may cause the SD-WAN leaf nodes to conservatively apply recommended SLA metrics (e.g., selecting a less degraded link than a link that better meets the recommended SLA metrics).

[0095] Each SD-WAN leaf node may select a path for a corresponding application and / or application group based on an intent-based SLA metric (716). For example, each SD-WAN leaf node may include an SLA path module 644 for establishing a path between endpoints on a path calculation domain. The SLA path module 644 of each SD-WAN leaf node may select a path for a corresponding application and / or application group that satisfies the intent-based SLA metric. The SLA path module 644 of each SD-WAN leaf node may select a path along one or more interconnected communication links that satisfies the intent-based SLA metric calculated by the intent module 642. For example, the SLA path module 644 may implement one or more shortest path algorithms as described above. In response to implementing the shortest path algorithm, the SLA path module 644 may determine whether each interconnected communication link satisfies the intent-based SLA metric. The SLA path module 644 may store the determined path and an indication of each interconnected communication link as to whether the link satisfies the intent-based SLA metric.

[0096] In some examples, each SD-WAN leaf node can also dynamically determine the shortest path based on per-hop metrics. For example, each SD-WAN leaf node can determine the shortest path when a user-configured intent model is changed (e.g., from an aggressive intent model to a fragile intent model). For example, the SD-WAN leaf node can be configured with an aggressive intent model for an application at an initial instance (e.g., T1). When the SD-WAN receives traffic from an application, the SD-WAN leaf node can select a corresponding link that satisfies the SLA metric of the aggressive intent model. In a subsequent instance (e.g., ΔT), the user can change the aggressive intent model to a conservative intent model for the application. When the SD-WAN leaf node receives traffic from an application, the SD-WAN leaf node can select a more deteriorated link that can satisfy the more conservative SLA metric of the fragile intent model.

[0097] Each SD-WAN leaf node may forward traffic for a particular application and / or application group over a selected path that satisfies the intent-based SLA metric (718). For example, the SD-WAN leaf node may send traffic for the corresponding application or application group over a link connecting SD-WAN leaf nodes that satisfies the intent-based SLA metric for the application or application group.

[0098] The techniques described herein may be implemented in hardware, software, firmware, or any combination thereof. The various features described as modules, units, or components may be implemented together in an integrated logic device, or separately as discrete but interoperable logic devices or other hardware devices. In some cases, various features of electronic circuits may be implemented as one or more integrated circuit devices, such as an integrated circuit chip or chipset.

[0099] If implemented in hardware, the invention may be directed to an apparatus such as a processor or an integrated circuit device such as an integrated circuit chip or chipset. Alternatively or additionally, if implemented in software or firmware, the techniques may be implemented at least in part by a computer-readable data storage medium comprising instructions that, when executed, cause a processor to perform one or more of the above methods. For example, a computer-readable data storage medium may store such instructions for execution by a processor.

[0100] The computer-readable medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may include computer data storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic or optical data storage media, etc. In some examples, the article of manufacture may include one or more computer-readable storage media.

[0101] In some examples, computer-readable storage media may include non-transient media. The term "non-transient" may indicate that the storage medium is not implemented in a carrier wave or propagating signal. In some examples, non-transient storage media may store data that changes over time (e.g., in RAM or cache).

[0102] The code or instructions may be software and / or firmware executed by a processing circuit that includes one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor" as used herein may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described in the present disclosure may be provided within a software module or a hardware module.

Claims

1. A system for network communication, comprising: Multiple network devices; A controller, wherein the controller is configured to: receiving one or more metrics of the plurality of network devices from the plurality of network devices, wherein the one or more metrics are associated with an application or a group of applications; calculating a recommended service level agreement (SLA) metric for the application or group of applications based on the one or more metrics received from the plurality of network devices; as well as sending the recommended SLA metric to each of the plurality of network devices so that each of the plurality of network devices calculates an intent-based SLA metric, the intent-based SLA metric being used to select a path for sending traffic from the application or application group, The intent-based SLA metric is calculated based on: the recommended SLA metric and an intent model, the intent model defining a tolerance level for applying the recommended SLA metric. 2 . The system of claim 1 , wherein to calculate the recommended SLA metric, the controller is configured to determine an aggregated mean of the one or more metrics received from the plurality of network devices.

3. A system according to claim 1 or 2, wherein to receive the metric, the controller is configured to receive the metric as part of a batch of multiple metrics.

4. The system of claim 1 , wherein a network device in the plurality of network devices is configured to: determining a metric for the network device, wherein the metric is associated with an application or a group of applications; sending the metric to the controller; receiving, from the controller, the recommended SLA metric associated with the application or group of applications; calculating an intent-based SLA metric based on: the recommended SLA metric and an intent model defining a tolerance level for applying the recommended SLA metric; as well as Based on the intent-based SLA metric, a path is selected for sending traffic from the application or group of applications.

5. The system according to claim 4, wherein in order to determine the metric for the network device, the network device is configured to determine a metric for one or more links connected to the network device, wherein in order to calculate the intent-based SLA metric, the network device is configured to calculate the intent-based SLA metric based on one or more characteristics of the one or more links, wherein the one or more links include wireless links, and The one or more characteristics of the wireless link include at least one of the following: latency, signal strength, coverage, data rate, bandwidth per unit area, frequency, and provider.

6. The system according to claim 4, wherein in order to determine the metric for the network device, the network device is configured to determine a metric for one or more links connected to the network device, wherein in order to calculate the intent-based SLA metric, the network device is configured to calculate the intent-based SLA metric based on one or more characteristics of the one or more links, wherein the one or more links include physical links, and The one or more characteristics of the physical link include at least link type, data rate support, transceiver, standard, provider, and signal strength.

7. The system of claim 4, wherein in order to select the path for sending traffic from the application or application group, the network device is configured to determine the shortest path based on at least one of the following: Dijkstra's algorithm, Bellman-Ford algorithm, breadth-first search BFS, depth-first search DFS, Floyd's algorithm, Johnson's algorithm, Kruskal's algorithm, Prim's algorithm, and algorithm.

8. The system of claim 4, wherein in order to determine the metric, the network device is configured to implement a measurement protocol, the measurement protocol comprising at least one of the following: real-time performance monitoring (RPM), a one-way active measurement protocol (OWAMP), a two-way active measurement protocol (TWAMP), active probing, passive probing, and a protocol deep packet inspection (DPI) engine.

9. The system of claim 4, wherein to send the metric, the network device is configured to send the metric as part of a batch of a plurality of metrics.

10. The system according to claim 4, wherein the network device is further configured to: An indication of whether one or more links connected to the network device satisfy the intent-based SLA metric is stored.

11. A method for network communication, comprising: receiving, by a controller, one or more metrics from a plurality of network devices, wherein the one or more metrics are associated with an application or a group of applications; Calculating, by the controller, a recommended service level agreement (SLA) metric for the application or application group based on the one or more metrics received from the plurality of network devices; as well as sending, by the controller, the recommended SLA metric to each of the plurality of network devices so that each of the plurality of network devices calculates an intent-based SLA metric, the intent-based SLA metric being used to select a path for sending traffic from the application or application group, The intent-based SLA metric is calculated based on: the recommended SLA metric and an intent model, the intent model defining a tolerance level for applying the recommended SLA metric.

12. The method of claim 11, wherein calculating the recommended SLA metric comprises determining an aggregated mean of the one or more metrics received from the plurality of network devices.

13. The method of any one of claims 11 to 12, wherein receiving the metric comprises receiving the metric as part of a batch of a plurality of metrics.

14. A method for network communication, comprising: determining, by a network device among a plurality of network devices, a metric of the network device, wherein the metric is associated with an application or a group of applications; sending, by the network device, the metric to a controller; receiving, by the network device from the controller, a recommended service level agreement (SLA) metric calculated by the controller and based at least in part on the metric sent by the network device, wherein the recommended service level agreement (SLA) metric is associated with the application or group of applications; Calculating, by the network device, an intent-based SLA metric based on: the recommended SLA metric and an intent model defining a tolerance level for applying the recommended SLA metric; as well as A path is selected, by the network device, for sending traffic from the application or group of applications based on the intent-based SLA metric.

15. The method according to claim 14, wherein determining the metric for the network device comprises determining a metric for one or more links connected to the network device, wherein calculating the intent-based SLA metric further comprises calculating the intent-based SLA metric based on one or more characteristics of the one or more links, wherein the one or more links include wireless links, and The one or more characteristics of the wireless link include at least one of the following: latency, signal strength, coverage, data rate, bandwidth per unit area, frequency, and provider.

16. The method according to claim 14, wherein determining the metric for the network device comprises determining a metric for one or more links connected to the network device, wherein calculating the intent-based SLA metric further comprises calculating the intent-based SLA metric based on one or more characteristics of the one or more links, wherein the one or more links include physical links, and The one or more characteristics of the physical link include at least link type, data rate support, transceiver, standard, provider, and signal strength.

17. The method of claim 14, wherein determining the path for sending traffic from the application or application group comprises determining the shortest path based on at least one of the following: Dijkstra's algorithm, Bellman-Ford algorithm, breadth-first search (BFS), depth-first search (DFS), Floyd's algorithm, Johnson's algorithm, Kruskal's algorithm, Prim's algorithm, and algorithm.

18. The method of claim 14, wherein determining the metric comprises implementing a measurement protocol, the measurement protocol comprising at least one of: real-time performance monitoring (RPM), a one-way active measurement protocol (OWAMP), a two-way active measurement protocol (TWAMP), active probing, passive probing, and a protocol deep packet inspection (DPI) engine.

19. The method of claim 14, wherein sending the metric comprises sending the metric as part of a batch of a plurality of metrics.

20. The method of claim 14, further comprising: An indication of whether one or more links connected to the network device satisfy the intent-based SLA metric is stored by the network device.

21. A computer-readable medium comprising computer-implementable instructions for causing a network device among a plurality of network devices to: determining a metric for the network device, wherein the metric is associated with an application or a group of applications; sending the metric to a controller; receiving from the controller a recommended SLA metric associated with the application or group of applications; calculating an intent-based SLA metric based on: the recommended SLA metric and an intent model, the intent model defining a tolerance level for applying the recommended SLA metric; as well as Based on the intent-based SLA metric, a path is selected for sending traffic from the application or group of applications.

Images