Wide area network link bandwidth monitoring

Abstract

Wide area network link bandwidth monitoring is disclosed. A network management system instructs a first network device to obtain one or more parameters related to a wireless local area network (WAN) link between the first network device and a second network device. The network management system executes a machine learning system configured to apply a machine learning model trained with link parameters to one or more parameters associated with the WAN link to predict a bandwidth of the link, thereby predicting a maximum bandwidth of the WAN link. The network management system outputs an indication of the predicted maximum bandwidth of the WAN link.

Description

[0001] Related applications

[0002] This application claims the benefit of U.S. application No. 18 / 346,009, filed on June 30, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to computer networks and, more particularly, to monitoring and / or managing network performance in a computer network. Background Art

[0004] A computer network is a collection of interconnected computing devices that can exchange data and share resources. Example computing devices include routers, switches, and other Layer 2 (L2) network devices that operate in Layer 2 (i.e., the data link layer) of the Open Systems Interconnection (OSI) reference model, and Layer 3 (L3) network devices that operate in Layer 3 (i.e., the network layer) of the OSI reference model. Network devices within a computer network typically include a control unit that provides control plane functions for the network device and forwarding components for routing or switching data units. Summary of the Invention

[0005] In general, the present disclosure describes techniques for predicting the maximum bandwidth of a wide area network (WAN) link between two network devices using a machine learning model. A pair of network devices (e.g., gateway routers) located at different sites in the WAN can be connected to each other on the WAN via one or more WAN links (e.g., logical paths) to transmit data between the pair of network devices. For example, a network management system can apply a machine learning model to one or more parameters obtained from the pair of network devices to more accurately determine the maximum bandwidth of the WAN link between the pair of network devices than traditional techniques, even when a third-party network, such as an Internet Service Provider (ISP) network, is interposed between the pair of network devices.

[0006] A network device may be able to detect that a WAN link connecting the network device to another network device in the WAN is performing poorly, for example, by detecting increased packet loss or by detecting that an application's bandwidth is being throttled. In response to detecting the poor performance of the WAN link, the network device may attempt to determine the cause of the poor performance of the WAN link by performing reactive actions, such as pinging another network device or performing a speed test. However, performing such actions may be counterproductive because performing pings and speed tests may add more load to an already overloaded WAN link. Furthermore, the network device may not be able to determine the actual cause of the poor performance of the WAN link by performing pings and speed tests after a WAN link problem has already occurred.

[0007] The maximum bandwidth of a WAN link can be information that is useful for diagnosing poor performance of a WAN link. For example, based on a comparison of the current bandwidth of the WAN link with the maximum bandwidth of the WAN link, a network device can determine whether poor performance of the WAN link is due to link exhaustion (e.g., overutilization of the link's available bandwidth). However, traditional techniques for determining maximum bandwidth (e.g., pings and speed tests) may not be sufficient to accurately determine the maximum bandwidth of a WAN link due to many external factors, such as other traffic on the WAN link, intermediary third-party networks between a pair of network devices, network conditions that depend on time and usage, etc. By more accurately determining the maximum bandwidth of a WAN link, the techniques of the present disclosure can provide greater granularity for determining whether performance degradation is due to WAN link exhaustion.

[0008] According to aspects of the present disclosure, a cloud-based network management system (NMS) that monitors WAN performance can proactively predict the maximum bandwidth of a WAN link connecting a pair of network devices. The NMS can use the predicted maximum bandwidth of the WAN link to determine the root cause of poor WAN link performance and recommend or invoke one or more remedial actions to address the poor WAN link performance.

[0009] The NMS can instruct one or more network devices in a pair of network devices located at different sites in a WAN and connected via a WAN link to perform a speed test on the WAN link to obtain one or more parameters related to the WAN link, such as the measured bandwidth of the WAN link. The NMS can apply a machine learning model trained using the link parameters to the one or more parameters to predict the bandwidth of the link, thereby predicting the maximum bandwidth of the WAN link. Therefore, the NMS can use the predicted maximum bandwidth of the WAN link to determine the root cause of poor WAN link performance and recommend or invoke one or more remedial measures to address the poor WAN link performance.

[0010] For example, the NMS may compare the bandwidth usage of a WAN link with the predicted maximum bandwidth of the WAN link to determine whether link exhaustion of the WAN link is the root cause of poor WAN link performance. Furthermore, if a pair of network devices are connected via multiple WAN links, when the WAN link between the pair of network devices is link exhaustion, the NMS may use the predicted maximum bandwidth of each of the multiple WAN links to determine whether another WAN link connecting the pair of network devices has available bandwidth. If the NMS determines that another WAN link connecting the pair of network devices has available bandwidth, the NMS may recommend or invoke remedial measures that switch network traffic from the link-exhausted WAN link to a WAN link with available bandwidth.

[0011] The disclosed techniques provide one or more technical advantages and specific improvements to the computer-related field of computer networking integrated into practical applications. These techniques may enable a cloud-based NMS to determine information that the NMS can use to identify the root cause of poor WAN link performance, for example, by identifying that the WAN link is link-depleted. Furthermore, the disclosed techniques may enable a network management system to recommend or initiate remedial actions to address poor WAN link performance, thereby improving WAN performance.

[0012] In some aspects, the technology described herein relates to a network management system comprising: a memory; and one or more processors in communication with the memory and configured to: instruct a first network device to obtain one or more parameters related to a wireless local area network (WAN) link between the first network device and a second network device; execute a machine learning system configured to apply a machine learning model trained using the link parameters to the one or more parameters related to the WAN link to predict a bandwidth of the link, thereby predicting a maximum bandwidth of the WAN link; and output an indication of the predicted maximum bandwidth of the WAN link.

[0013] In some aspects, the technology described herein relates to a method comprising: instructing, by one or more processors of a network management system, a first network device to obtain one or more parameters related to a wireless local area network (WAN) link between the first network device and a second network device; executing, by the one or more processors, a machine learning system configured to apply a machine learning model trained using the link parameters to the one or more parameters related to the WAN link to predict a bandwidth of the link, thereby predicting a maximum bandwidth of the WAN link; and outputting, by the one or more processors, an indication of the predicted maximum bandwidth of the WAN link.

[0014] In some aspects, the technology described herein relates to a computer-readable storage medium comprising instructions that, when executed, cause one or more processors of a network management system to: instruct a first network device to obtain one or more parameters associated with a wireless local area network (WAN) link between the first network device and a second network device; execute a machine learning system configured to apply a machine learning model trained using the link parameters to the one or more parameters associated with the WAN link to predict a bandwidth of the link, thereby predicting a maximum bandwidth of the WAN link; and output an indication of the predicted maximum bandwidth of the WAN link.

[0015] The details of one or more examples of the technology of the present disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the technology will become apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1A-Figure 1C is a block diagram illustrating an example network system including a network management system configured to predict the maximum bandwidth of a WAN link, in accordance with one or more techniques of this disclosure.

[0017] Figure 2 is a block diagram illustrating an example network device in accordance with techniques of this disclosure.

[0018] Figure 3 An example network management system configured in accordance with one or more techniques of this disclosure is shown.

[0019] Figure 4 is a block diagram illustrating an example implementation of an example network management system in accordance with techniques of this disclosure.

[0020] Figure 5 is a flow chart illustrating example operations performed by an example network management system.

[0021] Like reference numerals refer to like elements throughout the drawings and the description. DETAILED DESCRIPTION

[0022] Figure 1A-Figure 1C is a block diagram illustrating an example network system 100 including a network management system (NMS) 130 configured to predict the maximum bandwidth of a WAN link, in accordance with one or more techniques of this disclosure.

[0023] Figure 1A is a block diagram illustrating an example network system 100 according to the techniques of this disclosure. Figure 1A In the example of FIG, network system 100 includes networks 102A-102D (collectively referred to as "network 102"), which are configured to provide wide area network (WAN) connectivity to different customer networks 104A-104B ("customer networks 104") of an enterprise network. In some examples, network 102 is a service provider network. Although in Figure 1A In the example of FIG. 1 , the network system 100 is shown as including multiple interconnected networks 102 , but in other examples, the network system 100 may alternatively include a single network providing connectivity between customer networks 104 .

[0024] Network devices 110A-110I (collectively, “network devices 110”) of network 102 provide access to network 102 via customer edge devices 116A-116C (collectively, “CE devices 116”) to source devices 112A and 112B (collectively, “source devices 112”) and destination device 114 associated with customer network 104. The communication links between network devices 110 may be Ethernet, ATM, or any other suitable network connection.

[0025] The network device orchestrator (conductor, microservices orchestrator, orchestration framework) 120 is a centralized management and policy engine that provides orchestration, management, and zero-touch configuration for distributed network devices 110 while maintaining a network-wide multi-tenant service and policy data model. The network device orchestrator 120 can be considered an orchestrator. In some examples, the network device orchestrator 120 also provides monitoring and analytics for the network devices 110, while in other examples, only the monitoring and analytics for the network devices 110 and / or CE devices 116 are provided by the NMS 130. In some examples, the NMS 130 provides WAN assurance services to the network 102 and provides wireless assurance and / or wired assurance services to the customer networks 104. Figure 1A In the example of , NMS 130 includes a virtual network assistant 133 that can provide machine learning-based analysis of data collected by NMS 130 from network devices 110 of network 102 for WAN assurance services, and can provide machine learning-based analysis of data collected by NMS 130 from CE devices 116 or other customer devices within customer network 104 for wireless assurance and / or wired assurance services.

[0026] CE devices 116 and network devices 110 are discussed herein as routers for purposes of example. However, the techniques of the present disclosure may be implemented using any network device, such as a switch, a router, a gateway, or other suitable network device that can send and receive network traffic. For example, customer networks 104 may be networks of geographically separated sites of an enterprise network. Each customer network 104 may include additional customer devices, such as one or more non-edge switches, routers, hubs, gateways, security devices such as firewalls, intrusion detection and / or intrusion prevention devices, servers, computer terminals, laptops, printers, databases, wireless mobile devices such as cellular phones or personal digital assistants, wireless access points, bridges, cable modems, application accelerators, or Figure 1A Other network devices not shown. Figure 1A The configuration of the network system 100 shown is merely an example. For example, the network system 100 may include any number of customer networks 104. However, for ease of description, Figure 1A Only customer networks 104A-104B are shown.

[0027] 1. Networks 102 represent one or more publicly accessible computer networks owned and operated by one or more service providers. The service providers are typically large telecommunications entities or companies. Each network 102 is typically a large layer 3 (L3) computer network, where references to numbered layers refer to corresponding layers in the Open Systems Interconnection (OSI) model. Each network 102 is an L3 network in the sense that it natively supports L3 operations as described in the OSI model. Common L3 operations include operations performed in accordance with L3 protocols, such as the Internet Protocol (IP). L3 is also referred to as the "network layer" in the OSI model, and the term L3 may be used interchangeably with the phrase "network layer" in this disclosure.

[0028] Although not shown, each network 102 can be connected to one or more networks managed by other providers and can therefore form part of a large-scale public network infrastructure (e.g., the Internet). Therefore, customer networks 104 can be considered edge networks of the Internet. Each network 102 can provide computing devices within customer network 104 (e.g., source device 112 and destination device 114) access to the Internet and can allow computing devices within customer network 104 to communicate with each other.

[0029] Although additional network devices are not shown for ease of explanation, network system 100 may include additional network and / or computing devices, for example, one or more additional switches, routers, hubs, gateways, security devices such as firewalls, intrusion detection and / or intrusion prevention devices, servers, computer terminals, laptops, printers, databases, wireless mobile devices such as cellular phones or personal digital assistants, wireless access points, bridges, cable modems, application accelerators, or other network devices. Furthermore, although the elements of network system 100 are shown as being directly coupled, one or more additional network elements may be included along any communication link between network devices 110 such that the network elements of computer network system 100 are not directly coupled.

[0030] Each network 102 typically provides a number of residential and business services to customer networks 104, including residential and business-type data services (these data services are often referred to as "Internet services" because they allow access to the collection of publicly accessible networks known as the Internet), residential and business-type telephone and / or voice services, and residential and business-type television services.

[0031] In some examples, network device 110 comprises a packet-based router that employs a packet-based or flow-based routing scheme to forward packets according to a defined network path established by a centralized controller (e.g., a software-defined networking (SDN) controller) that performs path selection and traffic engineering. A given network device 110 (e.g., network device 110A) comprising a packet-based router operating as a network gateway for customer network 104A can establish multiple tunnels over a WAN with one or more other packet-based routers (e.g., network device 110I) operating as network gateways for other sites of an enterprise network (e.g., customer network 104B). As described herein, each packet-based router can collect data at the tunnel level, and the tunnel data can be retrieved by NMS 130 via an API or open configuration protocol, or the tunnel data can be reported to NMS 130 by a software agent or other module running on the packet-based router.

[0032] In other examples, the network device 110 includes a session-based router that employs a stateful, session-based routing scheme that enables each network device 110 to independently perform path selection and traffic engineering. The use of session-based routing can enable the network device 110 to avoid using a centralized controller (e.g., an SDN controller) to perform path selection and traffic engineering. In this way, the network device 110 may be more efficient and scalable for large networks where the use of an SDN controller is not feasible. In addition, the use of session-based routing can enable the network device 110 to avoid using tunnels, thereby saving significant network resources by eliminating the need to perform encapsulation and decapsulation at the tunnel endpoints. In some examples, the network device 110 implements session-based routing as Security Vector Routing (SVR) provided by Juniper Networks. A given network device 110 (e.g., network device 110A) including a session-based router operating as a network gateway for customer network 104A can establish multiple peer paths over a WAN with one or more other session-based routers (e.g., network device 110I) operating as network gateways for other sites of an enterprise network (e.g., customer network 104B). As described herein, each session-based router can include a software agent embedded in the session-based router that is configured to report path data collected at the peer path level to NMS 130.

[0033] A network session (also referred to herein as a "session") includes a forward packet flow originating from a first device and destined for a second device and / or a reverse packet flow originating from the second device and destined for the first device. The session can be bidirectional in that the session can include packets traveling in both directions between the first device and the second device (e.g., a forward packet flow and a reverse packet flow).

[0034] When, for example, network device 110A receives a packet of a flow originating from source device 112A and destined for destination device 114, network device 110A determines whether the packet belongs to a new session (e.g., is the "first" packet or the "leading" packet of the session). In some examples, network device 110A determines whether the source address, source port, destination address, destination port, and protocol of the first packet match an entry in a session table. If no such entry exists, network device 110A determines that the packet belongs to a new session and creates an entry in the session table. Additionally, if the packet belongs to a new session, network device 110A generates a session identifier for the session. The session identifier may include, for example, the source address and source port of source device 112A, the destination address and destination port of destination device 114, and the protocol used by the first packet. Network device 110A may use the session identifier to identify subsequent packets belonging to the session.

[0035] In some examples, network device 110 performs stateful routing for a session. This means that network device 110 forwards each packet of a forward packet flow for the session in sequence and along the same forward network path. As described herein, "the same" forward path means that the same network device 110 forms a segment or at least a portion between the device that originates the packet and the device to which the packet is destined (and not necessarily the entire network path between the device that originates the packet and the device to which the packet is destined). Additionally, network device 110 forwards each packet of a return flow for the session in sequence and along the same return network path. The forward network path for the forward packet flow for the session and the return network path for the return flow can be the same path, or different paths. By ensuring that each packet of the flow is forwarded in sequence and along the same path, network device 110 maintains state for the entire flow at each network device 110, thereby enabling the use of stateful packet services, such as deep packet inspection (DPI).

[0036] exist Figure 1AIn the example of FIG10 , a stateful routing session can be established from ingress network device 110A through intermediate network devices 110B-110H to egress network device 110I. In this example, network device 110A determines that a first packet is an unmodified packet and is the first packet of a new session. Network device 110A modifies the first packet to include metadata that specifies a session identifier (e.g., the original source address, source port, destination address, and destination port). Network device 110A replaces the header of the modified first packet to specify a source address that is the address of network device 110A, a source port that is the port through which network device 110A forwards the modified first packet to destination device 114, a destination address that is the address of the next hop to which network device 110A forwards the first packet (e.g., the address of network device 110B), and a destination port that is the port of the next hop to which network device 110A forwards the first packet (e.g., the port of network device 110B).

[0037] Network device 110A may further identify a network service associated with the session. For example, network device 110A may compare one or more of the source address, source port, destination address, or destination port of the session with a table of service address and port information to identify the service associated with the session. Examples of network services include Hypertext Transfer Protocol (HTTP), firewall services, proxy services, packet monitoring or metrics services, etc. For example, if the source port and / or destination port of the session is 80, the network device may determine that the session is associated with HTTP. In other examples, network device 110A may determine that one or more of the source address, source port, destination address, or destination port of the session belongs to an address or port block that indicates that a particular service is associated with the session.

[0038] In some examples, network device 110A uses the network service determined for the session to select a forwarding path for forwarding the first packet and each subsequent packet to destination device 114. In this manner, network device 110A can perform service-specific path selection to select a network path that best suits the service requirements. In contrast to network topologies in which an SDN controller performs path selection, each network device 110 performs path selection. Furthermore, the use of session-based routing enables each network device 110 to make routing decisions at the service or application level, in contrast to traditional network devices that can only make routing decisions at the flow level.

[0039] Network device 110A forwards the modified first packet to network device 110B. In addition, network device 110A stores the session identifier of the session so that when network device 110A receives subsequent packets of the session, it can identify the subsequent packets as belonging to the same session and forward them along the same path as the first packet.

[0040] Intermediate network device 110B receives the modified first packet and determines whether the modified first packet includes a portion of metadata specifying a session identifier. In response to determining that the modified first packet includes metadata specifying the session identifier, intermediate network device 110B determines that network device 110B is not an ingress device, such that network device 110B does not append the metadata specifying the session identifier.

[0041] As described above with respect to network device 110A, network device 110B determines whether a packet belongs to a new session (e.g., is the "first" packet or the "leading" packet of the session) by determining whether the source address, source port, destination address, destination port, and protocol of the first packet match an entry in the session table. If no such entry exists, network device 110B determines that the packet belongs to a new session and creates an entry in the session table. Additionally, if the packet belongs to a new session, network device 110B generates a session identifier for the session. The session identifier used by network device 110B to identify the session of the first packet may be different from the session identifier used by network device 110A to identify the same session of the first packet because each network device 110A, 110B generates the session identifier using the first packet's header source address, source port, destination address, and destination port, and this information is modified by each preceding network device 110 as each network device 110 forwards the first packet along the forwarding path. Additionally, each network device 110 may store the header information to identify the previous network device 110 (or “waypoint”) and the next network device 110 (or “waypoint”) so that each network device 110 may reconstruct the same forward path and reverse path for each subsequent packet of the session.

[0042] Network device 110B replaces the header of the modified first packet to specify a source address that is the address of network device 110B, a source port that is the port through which network device 110B forwards the modified first packet to destination device 114, a destination address that is the address of the next hop to which network device 110B forwards the first packet (e.g., the address of network device 110C), and a destination port that is the port of the next hop to which network device 110B forwards the first packet (e.g., the port of network device 110C). Network device 110B forwards the modified first packet to network device 110C. In addition, network device 110B stores the session identifier for the session so that upon receiving subsequent packets of the session, network device 110B can identify the subsequent packets as belonging to the same session and forward the subsequent packets along the same path as the first packet.

[0043] Subsequent intermediate network devices 110C-110H process the modified first packet in a similar manner as network devices 110A and 110B, causing network device 110 to forward subsequent packets of the session along the same path as the first packet. In addition, each network device 110 stores a session identifier for the session, which may include an identification of the previous network device 110 along the network path. Thus, each network device 110 can use the session identifier to forward packets of the reverse packet flow of the session back to source device 112A along the same network path.

[0044] Network device 110, which can forward packets of a forward packet flow for a session to the destination of the packet flow, is an egress or "terminal" network device. In the aforementioned example, network device 110I is the terminal network device because network device 110I can forward the packet to CE device 116C for forwarding to destination device 114. Network device 110I receives a modified first packet that includes metadata specifying a session identifier (e.g., the original source address, source port, destination address, and destination port). Network device 110I identifies the modified first packet as destined for a service terminating at network device 110I by determining that the destination source address and destination source port specified in the metadata of the modified first packet correspond to a destination reachable by network device 110I (e.g., via CE device 116C to destination device 114). Network device 110I restores the original first packet by removing the metadata from the modified first packet and modifying the header of the first packet to specify the original source address, source port, destination address, and destination port. Network device 110I forwards the recovered first packet to CE device 116C for forwarding to destination device 114 .

[0045] For additional information on session-based routing and SVR, see U.S. Patent No. 9,729,439, entitled “COMPUTER NETWORK PACKET FLOW CONTROLLER,” issued on August 8, 2017; U.S. Patent No. 9,729,682, entitled “NETWORK DEVICE AND METHOD FOR PROCESSING ASESSION USING APACKET SIGNATURE,” issued on August 8, 2017; U.S. Patent No. 9,762,485, entitled “NETWORK PACKET FLOW CONTROLLER WITH EXTENDED SESSION MANAGEMENT,” issued on September 12, 2017; U.S. Patent No. 9,871,748, entitled “ROUTER WITH OPTIMIZED STATISTICAL FUNCTIONALITY,” issued on January 16, 2018; and U.S. Patent No. 9,871,748, entitled “NAME-BASED ROUTING SYSTEM AND No. 9,985,883, entitled “LINK STATUS MONITORING BASED ON PACKETLOSS DETECTION,” issued on February 5, 2019; U.S. Patent No. 10,200,264, entitled “LINK STATUS MONITORING BASED ON PACKETLOSS DETECTION,” issued on April 30, 2019; U.S. Patent No. 10,277,506, entitled “STATEFUL LOADBALANCING IN A STATELESS NETWORK,” issued on October 1, 2019; U.S. Patent No. 10,432,522, entitled “NETWORK PACKET FLOW CONTROLLER WITH EXTENDED SESSION MANAGEMENT,” issued on October 1, 2019; and U.S. Patent No. 2020 / 0403890, entitled “IN-LINE PERFORMANCE MONITORING,” published on December 24, 2020, the entire contents of each of which are incorporated herein by reference in their entirety.

[0046] In some examples, to implement session-based routing, each network device 110 maintains a local repository of service and topology state information for each other network device 110. The service and topology state information includes services reachable from each network device 110 and the network topology used to reach these services from each network device. Each network device 110 can communicate changes in services reachable from the network device 110 and / or changes in the network topology used to reach the services from the network device to a central repository, such as a server. In addition, each network device 110 can receive service and topology state information for each other network device 110 in the computer network system 100 from the central repository.

[0047] In the foregoing example, network device 110A receives a packet, determines the session of the packet flow that includes the packet, determines the service associated with the session, and selects a network path for forwarding the packet. Network device 110A may use its local copy of the service and topology state information for each network device 110 to select a network path for forwarding the packet. For example, network device 110A may use the identified service associated with the packet and the network topology used to reach the identified service to select a network path that meets the service level agreement (SLA) requirements or other performance requirements of the service. Network device 110A may then forward the packet and subsequent packets of the flow along the selected path. In this manner, network device 110A may perform service-specific path selection because network device 110 may use criteria specific to the service associated with the packet to select a network path that best suits the service requirements. In other examples, network device 110A may select a network path that avoids (i.e., does not include) one or more paths with network performance anomalies, as discussed in further detail below, in order to select a network path with high network performance.

[0048] In some examples, the interfaces of network device 110 can be assigned to one or more "neighborhoods." A "neighborhood" is defined as a label applied to the interfaces of network device 110. Network devices 110 within the same neighborhood can form a peer relationship with each other. For example, each network device 110 with an interface with a neighborhood label applied can reach each other network device 110 with an interface with the same neighborhood label applied via a Layer 3 network. In some examples, one or more neighborhoods can be aggregated into a "zone." A zone is a logical grouping of one or more neighborhoods. Typically, an autonomous system (AS) (also referred to herein as an "authority") can be divided into one or more zones, each of which includes one or more neighborhoods.

[0049] In some examples, each network device 110 maintains a local repository of service and topology state information only for those other network devices 110 within the same neighborhood. In some examples, each network device 110 maintains a local repository of service and topology state information only for those other network devices 110 within the same zone of the neighborhood. As an example, each service provider network 102 can be considered a distinct "zone," where each subdomain within each service provider network 102 can be considered a neighborhood within that zone. In this example, each network device 110A and 110B within service provider network 102A can maintain service and topology state information only for each other, and not for network devices 110C-110I. Similarly, each network device 110D and 110C within service provider network 102B can maintain service and topology state information only for each other, and not for network devices 110A-110B or 110E-110I. In other examples, the administrator may assign one or more service provider networks 102 into one or more zones, one or more neighborhoods, or a combination of zones and neighborhoods that suits the needs of the network system 100 .

[0050] For additional information on the exchange of service and topology state information, see U.S. Patent Application Publication No. 2020 / 0366590, entitled “CENTRAL AUTHORITY FOR SERVICE AND TOPOLOGY EXCHANGE,” published on November 19, 2020; U.S. Patent Application Publication No. 2020 / 0366599, entitled “SOURCE-BASED ROUTING,” published on November 19, 2020; U.S. Patent Application Publication No. 2020 / 0366598, entitled “SERVICE AND TOPOLOGY EXCHANGE PROTOCOL,” published on November 19, 2020; U.S. Patent Application Publication No. 2020 / 0366589, entitled “ROUTING USING SEGMENT-BASED METRICS,” published on November 19, 2020; and U.S. Patent Application Publication No. 2020 / 0366589, entitled “ROUTING USING SEGMENT-BASED METRICS,” filed on July 31, 2018. ESTABLISHING COMMUNICATION RELATIONSHIPS BETWEEN COMMUNICATION INTERFACES IN AN ADMINISTRATIVE DOMAIN,” each of which is hereby incorporated by reference in its entirety.

[0051] According to the techniques described in this disclosure, NMS 130 is configured to predict the maximum bandwidth of a WAN link connecting a pair of network devices 110 in network 102. The WAN link connecting a pair of network devices 110 can be a logical path in network 102 (e.g., a peer path for session-based routers or a tunnel for packet-based routers) over which the pair of network devices 110 can communicate by sending and receiving data over the WAN link. The maximum bandwidth of the WAN link can be the maximum data transmission rate of the WAN link, which can be expressed in megabits per second, gigabits per second, etc.

[0052] NMS 130 is configured to predict a maximum bandwidth of a WAN link connecting a pair of network devices 110 in network 102, e.g., a WAN link connecting network device 110A to network device 110G in network 102, based on obtaining one or more parameters associated with the WAN link between the pair of network devices 110. The one or more parameters associated with the WAN link may indicate behavior of the WAN link during different levels of usage of the WAN link, e.g., behavior of the WAN link during periods of low usage and / or behavior of the WAN link during periods of high usage. The one or more parameters associated with the WAN link may indicate behavior of the WAN link during a maintenance window for one or more of the pair of network devices 110 connected via the WAN link. The one or more parameters associated with the WAN link may indicate behavior of the WAN link at different times of the day.

[0053] In some examples, the virtual network assistant 133 is configured to obtain one or more parameters related to a WAN link connecting a pair of network devices 110 based on performing a speed test on the WAN link. For example, the virtual network assistant 133 can instruct one or more of the pair of network devices connected via the WAN link to perform a speed test on the WAN link between the pair of network devices in order to obtain one or more parameters related to the WAN link. The one or more parameters related to the WAN link obtained from performing the speed test on the WAN link can include bandwidth measurements of the WAN link, such as the maximum bandwidth measured during the speed test, the minimum bandwidth measured during the speed test, and the like. The one or more parameters related to the WAN link obtained from performing the speed test on the WAN link can include bandwidth measurements of the WAN link, and can also include bandwidth measurements of interfaces of the network devices connected to the WAN link, such as the maximum bandwidth measured during the speed test, the minimum bandwidth measured during the speed test, and the like.

[0054] Virtual Network Assistant 133 can communicate with one or more of a pair of network devices connected via a WAN link to schedule the execution of a speed test for the WAN link. In some examples, Virtual Network Assistant 133 can schedule the execution of the speed test for the WAN link during different usage ratios or usage level periods of the WAN link, for example, during periods of high usage of the WAN link and / or periods of low usage of the WAN link. In some examples, Virtual Network Assistant 133 can schedule the execution of the speed test for the WAN link during a maintenance window for the first network device and / or the second network device, for example, during a time window immediately after an upgrade of the first network device and / or the second network device. In some examples, Virtual Network Assistant 133 can schedule the execution of the speed test for the WAN link at different times of the day. Additional information regarding the execution of tests for scheduling links and / or network devices is described in U.S. Provisional Patent Application No. 63 / 477,964, entitled “SYNTHETIC TESTING,” filed on December 30, 2022, and U.S. Provisional Patent Application No. 63 / 501,786, entitled “SYNTHETIC TESTING,” filed on May 12, 2023, the entire contents of both patent applications are incorporated herein by reference in their entirety.

[0055] The Virtual Network Assistant 133 can therefore obtain parameters related to the WAN link based on speed tests of the WAN link over a specific time period (e.g., one week, two weeks, one month, etc.). These parameters may include measurements of the bandwidth of the WAN link obtained during different usage ratios associated with the WAN link, measurements of the bandwidth of an interface of a network device obtained during different usage ratios associated with the WAN link, measurements of the bandwidth of the WAN link obtained during a maintenance window of the network device, and / or measurements of the bandwidth of the WAN link obtained at different times of the day.

[0056] Virtual network assistant 133 is configured to apply a machine learning model trained using link parameters to one or more parameters related to the WAN link to predict the bandwidth of the link, thereby predicting the maximum bandwidth of the WAN link. In other words, virtual network assistant 133 may input one or more parameters related to the WAN link obtained from the first network device into the machine learning model, and the machine learning model may output the predicted maximum bandwidth of the WAN link. NMS 130 may store the predicted maximum bandwidth in database 135.

[0057] Virtual network assistant 133 is configured to output an indication of a predicted maximum bandwidth of a WAN link between the first network device and the second network device. For example, virtual network assistant 133 can generate and output notifications, such as to a network administrator of the WAN, indicating the predicted maximum bandwidth of the WAN link.

[0058] Virtual network assistant 133 can use the predicted maximum bandwidth of a WAN link to detect problems (e.g., failures) with the WAN link at a given point in time and / or determine the root cause of a WAN link problem. For example, virtual network assistant 133 can detect a performance problem with a WAN link by measuring the bandwidth of the WAN link at a given point in time and comparing the measured bandwidth of the WAN link to the predicted maximum bandwidth of the WAN link. If virtual network assistant 133 determines that the measured bandwidth of the WAN link is less than the predicted maximum bandwidth of the WAN link by at least a predetermined amount, e.g., 90% less than the predicted maximum bandwidth of the WAN link, 95% less than the predicted maximum bandwidth of the WAN link, or the like, virtual network assistant 133 can determine that there is a performance problem associated with the WAN link and can determine that the poor performance of the WAN link is due to link exhaustion. Link exhaustion can occur when the available bandwidth of a link is overutilized, e.g., when the available bandwidth of a link is less than the amount of bandwidth required to satisfy the demand for network traffic exchanged on the link.

[0059] In some examples, NMS 130 may generate and output a notification with a recommendation to perform one or more remedial actions to resolve a WAN link issue, or may automatically invoke one or more remedial actions to resolve a WAN link issue. For example, if Virtual Network Assistant 133 determines that a WAN link connecting a pair of network devices 110 is a link-depleted link, Virtual Network Assistant 133 may determine whether there is another WAN link connecting the pair of network devices 110 that is not a link-depleted link. If Virtual Network Assistant 133 determines that there is another WAN link connecting the pair of network devices 110 that is not a link-depleted link, Virtual Network Assistant 133 may output a notification recommending switching network traffic from the link-depleted WAN link to a non-depleted WAN link, or may automatically execute a remedial action to switch network traffic from the link-depleted WAN link to a non-depleted WAN link.

[0060] The disclosed technology provides one or more technical advantages and practical applications. The technology can enable a cloud-based NMS 130 to more accurately determine the maximum bandwidth of a WAN link by using a machine learning model that predicts the maximum bandwidth of a WAN link based on parameters related to the WAN link obtained from speed tests performed on the WAN link.

[0061] The techniques of the present disclosure can improve the performance of WAN links by reducing the network load that may be placed on the WAN link to determine the maximum bandwidth of the WAN link and / or to diagnose performance issues associated with the WAN network. By being able to schedule the execution of speed tests of the WAN link, for example, during low-usage time periods of the WAN link, the techniques can also enable the NMS to reduce the impact of such speed tests on the performance of the WAN link. In addition, by determining the maximum bandwidth of the WAN link based on speed tests performed on the WAN link during previous time periods, the techniques of the present disclosure can avoid the need to perform reactive tests such as pings or speed tests after a performance issue associated with the WAN link is detected, thereby preventing additional network traffic from being added to a WAN link that may already be experiencing performance degradation.

[0062] Furthermore, the techniques of the present disclosure can identify the root cause of performance issues associated with WAN links and can recommend or invoke one or more remedial actions to resolve the performance issues associated with the WAN links. By determining the maximum bandwidth of the WAN link and by comparing the maximum bandwidth of the WAN link with the amount of bandwidth currently in use by the WAN link, the techniques of the present disclosure can determine whether the root cause of the performance issues associated with the WAN link is link exhaustion. Furthermore, the techniques of the present disclosure can recommend and / or invoke remedial actions, such as switching an application and / or session from the link-exhausted WAN link to another WAN link with available bandwidth capacity, thereby improving the performance of applications that transmit data from one network device to another network device.

[0063] Figure 1B yes Figure 1A In this example, a block diagram of a network system is provided with further example details. Figure 1B The NMS 130 is shown configured to operate according to an artificial intelligence / machine learning based computing platform that provides services from a wireless network 173 and a wired LAN 175 at the edge of the network. Figure 1B ) to the cloud-based application services 181 ( hosted by computing resources within data center 179 Figure 1B Full automation, insight, and assurance (e.g., Wireless Assurance, Wired Assurance, and WAN Assurance) for the entire network (see the far right of the diagram). Return to Reference Figure 1A , user devices 171 may include one or more source devices 112 and destination devices 114, and the wired LAN 175 hosting the wireless network 173 may include one or more customer networks 104 of the enterprise network.

[0064] As described herein, NMS 130 provides a comprehensive set of management tools and implements various technologies disclosed herein. Generally speaking, NMS 130 can provide a cloud-based platform for wireless network data acquisition, monitoring, activity logging, reporting, predictive analysis, network anomaly identification, and alert generation. For example, NMS 130 can be configured to proactively monitor and adaptively configure network system 100 to provide self-driving capabilities. Furthermore, VNA 133 includes a natural language processing engine to provide AI-driven support and troubleshooting, anomaly detection, AI-driven location services, and AI-driven RF optimization with reinforcement learning.

[0065] like Figure 1B As shown in the example of , the AI-driven NMS 130 can also configure, manage, monitor, and automate the supervision of a software-defined wide area network (SD-WAN) 177, which operates as an intermediate network that connects wireless networks 173 and wired LANs 175 to data centers 179 and application services 181. In general, the SD-WAN 177 is used to manage wireless networks 173 (e.g., branch or campus networks) (e.g., from Figure 1A The customer network 104, as a site of the enterprise network) provides seamless, secure, traffic engineering connections between the "spoke" network devices 187A and the "hub" network devices (routers) 187B of the edge wired network 175, further on the cloud stack towards the cloud-based application services 181. Return to reference Figure 1A , network devices 187A, 187B may include network device 110 operating as a network gateway for an enterprise network.

[0066] SD-WAN 177 typically operates and manages an overlay network 177 on an underlying physical wide area network (WAN) that provides access to geographically separated customer networks (e.g., Figure 1A In other words, the SD-WAN 177 can extend SDN capabilities and / or session-based routing or SVR capabilities to the WAN, allowing the network to decouple the underlying physical network infrastructure from the virtualized network infrastructure and applications, making it possible to configure and manage the network in a flexible and scalable manner.

[0067] In some examples, the underlying network devices of SD-WAN 177 can implement a stateful, session-based routing scheme, in which network devices 187A, 187B dynamically modify the contents of the original packet header originating from user device 171 to direct traffic to application service 181 along a selected path (e.g., peer path 189) without the use of tunnels and / or additional labels. In this manner, network devices 187A, 187B can be more efficient and scalable for large networks because the use of tunnel-free, session-based routing can enable network devices 187A, 187B to utilize significant network resources by eliminating the need to perform encapsulation and decapsulation at tunnel endpoints. Furthermore, in some examples, each network device (router) 187A, 187B can independently perform path selection and traffic engineering to control the flow of packets associated with each session, without requiring a centralized SDN controller for path selection and label distribution. In some examples, network devices 187A, 187B implement session-based routing as SVR provided by Juniper Networks.

[0068] For additional information on session-based routing and SVR, see U.S. Patent No. 9,729,439, entitled “COMPUTER NETWORK PACKET FLOW CONTROLLER,” issued on August 8, 2017; U.S. Patent No. 9,729,682, entitled “NETWORK DEVICE AND METHOD FOR PROCESSING A SESSION USING A PACKET SIGNATURE,” issued on August 8, 2017; U.S. Patent No. 9,762,485, entitled “NETWORK PACKET FLOW CONTROLLERWITH EXTENDED SESSION MANAGEMENT,” issued on September 12, 2017; U.S. Patent No. 9,871,748, entitled “ROUTER WITH OPTIMIZED STATISTICAL FUNCTIONALITY,” issued on January 16, 2018; and U.S. Patent No. 9,871,748, entitled “NAME-BASED ROUTING SYSTEM AND No. 9,985,883, entitled “LINK STATUS MONITORING BASED ON PACKETLOSS DETECTION,” issued on February 5, 2019, U.S. Patent No. 10,200,264, entitled “LINK STATUS MONITORING BASED ON PACKETLOSS DETECTION,” issued on April 30, 2019, U.S. Patent No. 10,277,506, entitled “STATEFUL LOADBALANCING IN A STATELESS NETWORK,” issued on October 1, 2019, U.S. Patent No. 10,432,522, entitled “NETWORK PACKET FLOW CONTROLLER WITH EXTENDED SESSION MANAGEMENT,” and U.S. Patent Application Publication No. 2020 / 0403890, entitled “IN-LINE PERFORMANCE MONITORING,” published on December 24, 2020, the entire contents of each of which are incorporated herein by reference in their entirety.

[0069] In some examples, the AI-driven NMS 130 can implement intent-based configuration and management of the network system 100, including the construction, presentation, and execution of intent-driven workflows for configuring and managing devices associated with the wireless network 173, the wired LAN network 175, and / or the SD-WAN 177. For example, declarative requirements express the desired configuration of network components without specifying the exact local device configuration and control flow. By utilizing declarative requirements, one can specify what should be accomplished, rather than how it should be accomplished. Declarative requirements can be contrasted with imperative instructions, which describe the exact device configuration syntax and control flow required to achieve the configuration. By utilizing declarative requirements rather than imperative instructions, the burden of determining the exact device configuration required to achieve the user / system's desired outcome is reduced for the user and / or user's system. For example, when utilizing a variety of different types of devices from different vendors, specifying and managing exact imperative instructions to configure each device in the network is often difficult and cumbersome. The types and variety of devices in a network can change dynamically as new devices are added and device failures occur. Managing a diverse and diverse set of devices from different vendors with varying configuration protocols, syntaxes, and software versions to configure a collaborative network of devices is often difficult to implement. Thus, by requiring only the user / system to specify declarative requirements that specify desired outcomes applicable to a variety of different types of devices, management and configuration of network devices becomes more efficient. Further example details and techniques of intent-based network management systems are provided in U.S. Patent No. 10,756,983, entitled “Intent-based Analytics,” and U.S. Patent No. 10,992,543, entitled “Automatically generating an intent-based network model of an existing computer network,” both of which are incorporated herein by reference.

[0070] According to the techniques described in this disclosure, NMS 130 is configured to predict the maximum bandwidth of a WAN link between a pair of network devices, for example, the maximum bandwidth of a logical path 189 (e.g., a peering path or tunnel) between network devices 187A and 187B in SD-WAN 177. In some examples, one or more intermediate third-party networks (e.g., one or more Internet Service Provider (ISP) networks, access networks, or transport networks) may serve as the basis for logical path 189 to provide connectivity between network devices (routers) 187A and 187B. NMS 130 may communicate with network device (router) 187A and / or network device (router) 187B to schedule a speed test for execution within a specified time period (e.g., one month) to obtain parameters from the speed test, such as the measured bandwidth of logical path 189 and / or the measured bandwidth of interfaces of network device (router) 187A and / or network device (router) 187B connected to logical path 189.

[0071] NMS 130 can schedule execution of the speed test for logical path 189 during different usage ratios or usage level time periods of logical path 189, for example, during a high usage time period of logical path 189 and / or during a low usage time period of logical path 189. In some examples, NMS 130 can schedule execution of the speed test for logical path 189 during a maintenance window of network device (router) 187A and / or network device (router) 187B, for example, during a time window immediately after network device (router) 187A and / or network device (router) 187B is upgraded. In some examples, NMS 130 can schedule execution of the speed test for logical path 189 at different times of the day.

[0072] The NMS 130 is configured to apply the machine learning model trained using the parameters of the link to one or more parameters related to the logical path 189 to predict the bandwidth of the link, thereby predicting the maximum bandwidth of the logical path 189. In other words, the NMS 130 may input one or more parameters related to the logical path 189 into the machine learning model, and the machine learning model may output the predicted maximum bandwidth of the logical path 189, and the NMS 130 may store the predicted maximum bandwidth in the database 135.

[0073] The NMS 130 is configured to output an indication of the predicted maximum bandwidth of the logical path 189 between the network devices 187A, 187B. For example, the virtual network assistant 133 may generate and output notifications, e.g., to a network administrator of the SD-WAN 177, indicating the predicted maximum bandwidth of the logical path 189.

[0074] In some examples, NMS 130 is configured to use the predicted maximum bandwidth of logical path 189 to determine a failure in logical path 189 and / or to determine a root cause of a failure in logical path 189. For example, virtual network assistant 133 can detect a performance problem with logical path 189 by measuring the bandwidth of logical path 189 at a given point in time and comparing the measured bandwidth of logical path 189 to the predicted maximum bandwidth of logical path 189. If virtual network assistant 133 determines that the measured bandwidth of the WAN link is less than the predicted maximum bandwidth of the WAN link by at least a predetermined amount, virtual network assistant 133 can determine that there is a performance problem associated with logical path 189 and can determine that the poor performance of logical path 189 is due to link exhaustion.

[0075] In some examples, NMS 130 is configured to generate and output a notification with a recommendation to perform one or more remedial actions to resolve the failure of logical path 189 and / or automatically invoke one or more remedial actions to resolve the failure of logical path 189. For example, if virtual network assistant 133 determines that logical path 189 is exhausted, virtual network assistant 133 may output a recommendation to avoid forwarding network traffic via logical path 189 and / or switch one or more applications that are using logical path 189 to use another logical path between network devices 187A, 187B that has available bandwidth, or may automatically switch one or more applications from using logical path 189 to use another logical path between network devices 187A, 187B.

[0076] Figure 1C It shows Figure 1B 1. A block diagram of the network system 100 is provided for further example details. Specifically, Figure 1C Shown Figure 1B Figure 1 shows an example SD-WAN deployment architecture for SD-WAN 177. In the example shown, SD-WAN 177 includes spoke network devices (routers) 187A within a branch office, which are connected to hub network devices (routers) 187B within a data center via logical paths 189 through an underlying physical WAN (e.g., MPLS network 188). SD-WAN 177 also includes hosted or software-as-a-service (SaaS) applications.

[0077] When troubleshooting SD-WAN issues, it may be helpful to separate the problem into three parts: 1) the branch office, 2) the logical path (e.g., peer path or tunnel) through the WAN (e.g., MPLS, LTE, or broadband network), and 3) application services, including applications hosted internally (e.g., in a data center) and SaaS applications. The NMS 130 can be configured to track the temporal connection topology of these three segments for each customer deployment and also detect various types of user-impacting issues in the Virtual Network Assistant 133. By combining the connection topology with the corresponding events occurring in each segment, the Virtual Network Assistant 133 of the NMS 130 is able to pinpoint the location and root cause of different user-impacting SD-WAN issues. Examples of user-impacting issues for the branch office segment may include device health, cable failures, and configuration issues (e.g., maximum transmission unit (MTU)). Examples of user-impacting issues for the logical path segment may include link connectivity and link performance degradation. Examples of user-impacting issues for the application services segment may include service reachability and service performance.

[0078] According to the techniques described in this disclosure, the NMS 130 is configured to predict the maximum bandwidth of a WAN link between a pair of network devices, for example, the maximum bandwidth of a logical path 189 (e.g., a peer path or tunnel) between network devices 187A, 187B in the SD-WAN 177. The NMS 130 can communicate with the network device (router) 187A and / or the network device (router) 187B to schedule the execution of a speed test within a specified time period (e.g., one month) to obtain parameters from the speed test, such as the measured bandwidth of the logical path 189 and / or the measured bandwidth of the interfaces of the network device 187A (router) and / or the network device (router) 187B connected to the logical path 189.

[0079] NMS 130 can schedule execution of the speed test for logical path 189 during different usage ratios or usage level time periods of logical path 189, for example, during a high usage time period of logical path 189 and / or during a low usage time period of logical path 189. In some examples, NMS 130 can schedule execution of the speed test for logical path 189 during a maintenance window of network device (router) 187A and / or network device (router) 187B, for example, during a time window immediately after network device (router) 187A and / or network device (router) 187B is upgraded. In some examples, NMS 130 can schedule execution of the speed test for logical path 189 at different times of the day.

[0080] The NMS 130 is configured to apply the machine learning model trained using the link parameters to one or more parameters related to the logical path 189 to predict the bandwidth of the link, thereby predicting the maximum bandwidth of the logical path 189. In other words, the NMS 130 may input one or more parameters related to the logical path 189 into the machine learning model, and the machine learning model may output the predicted maximum bandwidth of the logical path 189, and the NMS 130 may store the predicted maximum bandwidth in the database 135.

[0081] The NMS 130 is configured to output an indication of the predicted maximum bandwidth of the logical path 189 between the network devices 187A, 187B. For example, the virtual network assistant 133 may generate and output notifications, e.g., to a network administrator of the SD-WAN 177, indicating the predicted maximum bandwidth of the logical path 189.

[0082] In some examples, NMS 130 is configured to use the predicted maximum bandwidth of logical path 189 to determine a failure in logical path 189 and / or to determine a root cause of a failure in logical path 189. For example, virtual network assistant 133 can detect a performance problem with logical path 189 by measuring the bandwidth of logical path 189 at a given point in time and comparing the measured bandwidth of logical path 189 to the predicted maximum bandwidth of logical path 189. If virtual network assistant 133 determines that the measured bandwidth of the WAN link is less than the predicted maximum bandwidth of the WAN link by at least a predetermined amount, virtual network assistant 133 can determine that there is a performance problem associated with logical path 189 and can determine that the poor performance of logical path 189 is due to link exhaustion.

[0083] In some examples, NMS 130 is configured to generate and output a notification with a recommendation to perform one or more remedial actions to address a failure of logical path 189 and / or automatically invoke one or more remedial actions to address a failure of logical path 189. For example, if virtual network assistant 133 determines that logical path 189 is exhausted, virtual network assistant 133 may output a recommendation to avoid forwarding network traffic via logical path 189 and / or switch one or more applications that are using logical path 189 to use another logical path between network devices 187A, 187B that has available bandwidth, or may automatically switch one or more applications from using logical path 189 to use another logical path between network devices 187A, 187B.

[0084] Figure 2 is a block diagram illustrating an example network device 200 according to the techniques of this disclosure. In general, the network device 200 may be Figure 1A A network device 110 or Figure 1B and Figure 1C 187B. In this example, network device 200 includes interface cards 226A-226N ("IFCs 226") that receive packets via input links 228A-228N ("input links 228") and transmit packets via output links 230A-230N ("output links 230"). IFCs 226 are typically coupled to links 228, 230 via a plurality of interface ports. Network device 200 also includes a control unit 202 that determines a route for received packets and forwards the packets accordingly via IFCs 226.

[0085] The control unit 202 may include a routing engine 204 and a packet forwarding engine 222. The routing engine 204 serves as the control plane of the network device 200 and includes an operating system that provides a multi-tasking operating environment to execute multiple concurrent processes. The routing engine 204 is coupled to other routers (e.g., Figure 1A 110) to establish and maintain a computer network (e.g., Figures 1A to 1C The network system 100 is used to transmit network traffic between one or more customer devices. The routing protocol process (RPD) 208 of the routing engine 204 executes software instructions to implement one or more control plane network protocols 212. For example, the protocols 212 may include one or more routing protocols, such as the Internet Group Management Protocol (IGMP) 221 and / or the Border Gateway Protocol (BGP) 220, which are used to exchange routing information with other routing devices and update the routing information base (RIB) 206; the Multiprotocol Label Switching (MPLS) protocol 214 and other routing protocols. The protocols 212 may also include one or more communication session protocols 223, such as TCP, UDP, TLS or ICMP. The protocols 212 may also include one or more performance monitoring protocols, such as BFD 225.

[0086] The RIB 206 can describe the topology of the computer network in which the network device 110 is located, and can also include the routes through the shared tree in the computer network. The RIB 206 describes the various routes in the computer network and the appropriate next hop for each route, i.e., the adjacent routing devices along each route. The routing engine 204 analyzes the information stored in the RIB 206 and generates forwarding information for the forwarding engine 222, which is stored in the forwarding information base (FIB) 224. The FIB 224 can associate, for example, a network destination with a specific next hop and a corresponding IFC 226 and the physical output port of an output link 230. The FIB 224 can be a radix tree, a series of tables, a complex database, a list of links, a radix tree, a database, a flat file, or various other data structures programmed into a dedicated forwarding chip.

[0087] The FIB 224 may also include a lookup structure. For a given keyword, such as an address, the lookup structure may provide one or more values. In some examples, the one or more values ​​may be one or more next hops. The next hop may be implemented as microcode that, when executed, performs one or more operations. One or more next hops may be "chained" such that a set of chained next hops, when executed, perform a set of operations for each different next hop. Examples of such operations may include applying one or more services to a packet, dropping a packet, and / or forwarding the packet using an interface and / or interfaces identified by the one or more next hops.

[0088] Session information 235 stores information used to identify a session. In some examples, session information 235 is in the form of a session table. For example, service information 232 includes one or more entries that specify a session identifier. In some examples, the session identifier includes one or more of a source address, a source port, a destination address, a destination port, or a protocol associated with a forward flow and / or a reverse flow of the session. As described above, when routing engine 204 receives a request from a server, Figure 1A client device (e.g., source device 112A) and go to Figure 1A When routing engine 204 receives a packet in a forward packet flow destined for another client device (e.g., destination device 114), routing engine 204 determines whether the packet belongs to a new session (e.g., is the "first" packet or the "first" packet of the session). To determine whether the packet belongs to a new session, routing engine 204 determines whether session information 235 includes an entry corresponding to the source address, source port, destination address, destination port, and protocol of the first packet. If an entry exists, the session is not a new session. If no entry exists, the session is new, and routing engine 204 generates a session identifier for the session and stores the session identifier in session information 235. Thereafter, routing engine 204 can use the session identifier stored in session information 235 for the session to identify subsequent packets belonging to the same session.

[0089] Service information 232 stores information that routing engine 204 can use to identify services associated with a session. In some examples, service information 232 takes the form of a service table. For example, service information 232 includes one or more entries specifying a service identifier and one or more of a source address, source port, destination address, destination port, or protocol associated with the service. In some examples, routing engine 204 can query service information 232 using one or more of the source address, source port, destination address, destination port, or protocol for a received packet's session to determine the service associated with the session. For example, routing engine 204 can determine the service identifier based on a correspondence between the source address, source port, destination address, destination port, or protocol in service information 232 and the source address, source port, destination address, destination port, or protocol specified by the session identifier. Based on the service associated with the packet, routing engine 204 retrieves one or more service policies 234 corresponding to the identified service. Service policies can include, for example, path failover policies, Dynamic Host Configuration Protocol (DHCP) tagging policies, traffic engineering policies, and prioritization of network traffic associated with the session. Routing engine 204 applies one or more service policies 234 corresponding to the services associated with the packet to the packet.

[0090] In some examples, network device 200 can include a session-based router that employs a stateful, session-based routing scheme that enables routing engine 204 to independently perform path selection and traffic engineering. The use of session-based routing can enable network device 200 to avoid using a centralized controller (e.g., an SDN controller) to perform path selection and traffic engineering, and avoid the use of tunnels. In some examples, network device 200 can implement session-based routing as Secure Vector Routing (SVR) provided by Juniper Networks, Inc. In instances where network device 200 includes a session-based router that operates as a network gateway for a site of an enterprise network, network device 200 can establish multiple peer paths over an underlying physical WAN with one or more other session-based routers that operate as network gateways for other sites of the enterprise network.

[0091] Although primarily described herein as a session-based router, in other examples, network device 200 may comprise a packet-based router in which routing engine 204 employs a packet-based or flow-based routing scheme to forward packets according to a defined network path established, for example, by a centralized controller that performs path selection and traffic engineering. In the event that network device 200 comprises a packet-based router operating as a network gateway for a site of an enterprise network, network device 200 may establish multiple tunnels over the underlying physical WAN with one or more other packet-based routers operating as network gateways for other sites of the enterprise network.

[0092] In accordance with the techniques of this disclosure, the control unit 202 of the network device 200 is configured to perform a speed test of a WAN link between the network device 200 and other network devices in the SD-WAN 177 and to collect data associated with the speed test. For example, a speed test agent 238 may be executed at the network device 200 to perform a speed test of the WAN link between the network device 200 and other network devices in the SD-WAN 177. The speed test agent 238 may collect speed test data, such as the maximum bandwidth of the WAN link measured during the speed test and the measured maximum bandwidth of an interface (e.g., one of the IFCs 226) connected to the WAN link, and may report the speed test data to the NMS 130.

[0093] Speed ​​test agent 238 may perform a speed test of the WAN link to measure the maximum bandwidth of the WAN link during the speed test and / or measure the maximum bandwidth of an interface of network device 200 connected to the WAN link via any suitable speed test technique. For example, speed test agent 238 may send a series of very large messages via the WAN link to another network device connected to network device 200 via the WAN link, and may measure the end-to-end throughput of the WAN link while transmitting the series of messages to measure the maximum bandwidth of the WAN link during the speed test.

[0094] In some examples, the speed test agent 238 can schedule a speed test for the WAN link based on the usage level of the WAN link. For example, the speed test agent 238 can schedule a speed test during a high usage period of the WAN link and during a low usage period of the WAN link. The speed test agent 238 can determine the usage level of the WAN link by any suitable technique, such as based on the number of sessions in the WAN link, the number of applications using the WAN link, the amount of bandwidth of the WAN link being used, etc. For example, if the amount of bandwidth of the WAN link being used is below a specified low usage threshold, the speed test agent 238 can determine that the usage level of the WAN link is low, and if the amount of bandwidth of the WAN link being used is above a specified high usage threshold, the speed test agent 238 can determine that the usage level of the WAN link is high.

[0095] In some examples, the speed test agent 238 can schedule a speed test of the WAN link during a maintenance window of the network device (router device) 200, where the maintenance window can be a time window immediately after the network device 200 is upgraded, or a time window immediately after the network device 200 is rebooted. Such a time window can be a 30-minute time window immediately after the network device 200 is upgraded or rebooted, a 1-hour time window immediately after the network device 200 is upgraded or rebooted, and so on.

[0096] In some examples, the speed test agent 238 can schedule a speed test for a WAN link at a specific time of day. The specific time of day can be based on the occurrence of a network congestion event on the WAN link. A network congestion event can occur on a WAN link when the WAN link experiences a large amount of network traffic and becomes overloaded. For example, a network congestion event can occur on a WAN link when the network traffic on the WAN link consumes substantially all of the bandwidth of the WAN link, e.g., when the network traffic on the WAN link consumes 99% or more of the maximum bandwidth of the WAN link.

[0097] The speed test agent 238 can detect the occurrence of a network congestion event on a WAN link by monitoring the performance of the WAN link, for example, by monitoring WAN link metrics (e.g., latency, jitter, and loss), application metrics (e.g., round-trip time for retransmissions and TCP acknowledgments), metrics regarding the correlation of degraded link performance with high bandwidth usage, metrics regarding long-term link stability, and / or the quality of service of the WAN link (e.g., mean opinion score). For example, if the speed test agent 238 determines that the latency, jitter, and / or loss of the WAN link suddenly exceeds a specified threshold within a specified time period, the speed test agent 238 can determine that a network congestion event has occurred on the WAN link.

[0098] The speed test agent 238 can determine a time of day when a network congestion event is likely to occur on the WAN link based on the occurrence of the network congestion event on the WAN link, and can schedule a speed test for the WAN link at a time of day before the network congestion event is likely to occur, i.e., a specified amount of time (e.g., 1 hour) before the network congestion event is likely to occur. For example, if the speed test agent 238 detects the occurrence of a network congestion event at 12:12 p.m. every day, the speed test agent 238 can determine that the network congestion event is likely to occur at 12:12 p.m. every day, and can therefore schedule a speed test for the WAN link at a time of day before the network congestion event is likely to occur, i.e., a specified amount of time before the network congestion event is likely to occur, e.g., by scheduling a speed test at 11:12 a.m. every day.

[0099] Figure 3 An example network management system (NMS) 300 configured according to one or more techniques of the present disclosure is shown. NMS 300 may be used to implement, for example, Figure 1A-Figure 1C NMS 300 can be responsible for monitoring and managing the network 102. Figure 1A Network devices 110A-110I, Figure 1B-1C Network devices 187A, 187B or Figure 2 One or more of the network devices 200.

[0100] In this example, NMS 300 receives path data collected by network devices 110A-110N. Path data may include statistics and data samples at the logical path (e.g., peer path or tunnel) level, such as telemetry data and data extracted from messages and / or counters. In some examples, path data may also include detailed information about clients connected to network device 110. In further examples, path data may include event-driven path data reported in response to the occurrence of specific events at network device 110.

[0101] NMS 300 uses the path data to calculate one or more SLE metrics to monitor the health of the logical paths from network devices 110 through the underlying physical WAN and detect network failures and performance degradation that may affect the user experience. Figure 1A-Figure 1C A server that is part of a microservice cloud infrastructure within the network system 100 or accessible by the network system 100.

[0102] In some examples, in addition to monitoring network devices 110, NMS 300 is also responsible for monitoring and managing one or more wireless or wired networks (e.g., Figure 1BIn this example, the NMS 300 also receives data from the access point to the user equipment (e.g., Figure 1B The NMS 300 collects data from user devices 171 and analyzes the data for cloud-based management of the wireless network. In this way, a single NMS 300 can be used to manage network devices 110, which may include virtualized network devices (e.g., software-based routers executed on virtual machines or containers), and wireless networks for an end-to-end WAN assurance system visible via a single cloud-based WAN assurance portal.

[0103] NMS 300 includes a communication interface 330, one or more processors 306, a user interface device 310, memory 312, and link data 318. The various components are coupled together via a bus 314, over which they can exchange data and information. Processor 306 executes software instructions, such as those defining software or a computer program, stored in a computer-readable storage medium (e.g., memory 312), such as a non-transitory computer-readable medium including a storage device (e.g., a disk drive or optical drive) or memory (e.g., flash memory or RAM), or any other type of volatile or non-volatile memory, that stores instructions for causing one or more processors 306 to perform the techniques described herein. User interface device 310 can be any suitable device for outputting data, such as a display device that outputs one or more user interfaces (e.g., one or more graphical user interfaces).

[0104] The communication interface 330 may include, for example, an Ethernet interface. The communication interface 330 couples the NMS 300 to a network and / or the Internet, for example, Figure 1A The communication interface 330 includes a receiver 332 and a transmitter 334, through which the NMS 300 transmits data to / from any network device 110 and / or any network device 110 formed as shown. Figure 1A The NMS 300 may receive and send data and information to / from any other device or system that is part of the illustrated network 102 or 104. The data and information received by the NMS 300 may include, for example, network performance data, SLE-related or event log data received from the network devices 110 and used by the NMS 300 to remotely monitor the performance of the network devices 110 and the network 102. In some examples, the NMS may also send data to any network device 110 via the communication interface 330 to remotely manage the network 102.

[0105] The memory 312 includes one or more devices configured to store programming modules and / or data related to the operation of the NMS 300. For example, the memory 312 may include a computer-readable storage medium, such as a non-transitory computer-readable medium including a storage device (e.g., a magnetic disk drive or an optical disk drive) or memory (e.g., flash memory or RAM) or any other type of volatile or non-volatile memory that stores instructions to cause the one or more processors 306 to perform the techniques described herein.

[0106] In this example, memory 312 includes API 320, Virtual Network Assistant (VNA) / AI engine 350 including WAN link performance engine 352 and root cause analysis engine 370, and ML model 356. NMS 300 may also include any other programming modules, software engines, and / or interfaces configured for remote monitoring and management of network devices 110, including remote monitoring and management of any network device 110. NMS 300 may also include any other programming modules, software engines, and / or interfaces configured for remote monitoring and management of a wireless network, including remote monitoring and management of any access point.

[0107] The VNA / AI engine 350 analyzes link data 318 received from network devices 110 and its own data to identify when an unexpected or abnormal state is encountered in one of the networks 102. For example, the VNA / AI engine 350 may use a root cause analysis module 354 to identify the root cause of any unexpected or abnormal state. In some examples, the root cause analysis module 354 utilizes artificial intelligence-based techniques to help identify the root cause of any poor SLE metrics and / or poor network performance at one or more networks 102. Furthermore, the VNA / AI engine 350 may automatically invoke one or more corrective actions designed to address the identified root cause(s) of the one or more poor SLE metrics and / or poor performance. Examples of corrective actions that may be automatically invoked by the VNA / AI engine 350 may include, but are not limited to, invoking an API 320 to restart one or more network devices 110. Corrective actions may also include switching applications from WAN links to different paths and / or interfaces, restarting switches and / or routers, invoking the download of new software to network devices, switches, or routers, and the like. These corrective actions are provided for example purposes only, and the present disclosure is not limited in these respects. If automatic corrective actions are not available or are insufficient to address the root cause, the VNA / AI engine 350 can proactively provide notifications that include recommended corrective actions for IT personnel to take to resolve the network error.

[0108] In some examples, the VNA / AI engine 350 can construct, train, apply, and retrain supervised and / or unsupervised ML models, such as the ML model 356, to predict the maximum bandwidth of a WAN link based at least in part on one or more parameters related to the WAN link. The ML model 346 can be an artificial neural network, such as a convolutional neural network (CNN) or a recurrent neural network (RNN), or any other model trained using machine learning to predict the maximum bandwidth of a WAN link based at least in part on one or more parameters related to the WAN link.

[0109] The NMS 300 or another computing system can train the ML model 356 using any suitable machine learning technique, such as supervised learning, unsupervised learning, semi-supervised training, reinforcement learning, etc., to predict the maximum bandwidth of the WAN link based at least in part on one or more parameters associated with the WAN link. In some examples, the NMS 300 or another computing system can train the ML model 356 using training data that includes parameters associated with a plurality of different WAN links (of the network 102 and / or other networks) obtained over a previous time period (e.g., over a previous year or multiple years, over a previous month or multiple months, etc.). For example, for each of the plurality of different WAN links, the training data can include a set of corresponding inputs (e.g., a set of parameters associated with the corresponding WAN link) as training examples and a desired output, such as an associated maximum bandwidth of the corresponding WAN link, to train the ML model 356 to predict the maximum bandwidth of the WAN link based on the set of parameters associated with the WAN link for each WAN link.

[0110] The set of parameters associated with the corresponding WAN link in the training data may include parameters of the corresponding WAN link under various network conditions, such as different WAN link usage levels, different times of the day, different days of the week, and / or any other relevant network conditions. These parameters may include the maximum bandwidth of the WAN link during one or more low-usage periods of the WAN link and / or one or more high-usage periods of the WAN link, WAN link metrics (e.g., latency, jitter, and loss) of the WAN link during the various WAN link usage periods, application metrics (e.g., TCP acknowledgment retransmission and round-trip time) during the various WAN link usage periods, interface bandwidth of one or more network devices connected via the WAN link, number of sessions and / or session duration during the various WAN link usage periods, amount of data received and / or transmitted by the one or more network devices during the various WAN link usage periods, and / or any combination thereof.

[0111] The VNA / AI engine 350 is configured to communicate with one or more of the pair of network devices 110 connected via a WAN link to schedule execution of a speed test, thereby obtaining one or more parameters related to the WAN link based on the speed test. In some examples, the one or more parameters related to the WAN link obtained from execution of the speed test of the WAN link may include one or more measurements of the bandwidth of the WAN link between the pair of network devices 110, such as one or more measurements of the data transmission rate (e.g., expressed in megabits per second, gigabits per second, etc.) between the pair of network devices 110 on the WAN link. In some examples, the one or more parameters related to the WAN link obtained from execution of the speed test of the WAN link may also include one or more measurements of the bandwidth of interfaces of one or more of the network devices in the pair of network devices 110 connected via the WAN link.

[0112] In some examples, the VNA / AI engine 350 can schedule the execution of speed tests on the WAN link connecting a pair of network devices 110 across time periods of different usage levels of the WAN link. The VNA / AI engine 350 can schedule the speed tests on the WAN link to be executed during low usage time periods of the WAN link, which can be time periods when there is no traffic or relatively low traffic on the WAN link, and the speed tests on the WAN link to be executed during high usage time periods of the WAN link, which can be time periods when there is relatively high traffic on the WAN link.

[0113] The VNA / AI engine 350 can communicate with one or more of a pair of network devices 110 connected via a WAN link to determine low-usage periods of the WAN link and / or high-usage periods of the WAN link, and schedule the execution of a speed test for the WAN link during these periods. In some examples, the VNA / AI engine 350 can specify one or more thresholds to one or more of the pair of network devices that indicate that the WAN link is in a low-usage period, such as a specified threshold for the amount of data received and / or transmitted by one or more of the pair of network devices 110, and if the amount of data received and / or transmitted by one or more of the pair of network devices 110 is below the specified threshold, the VNA / AI engine 350 can determine that the WAN link is in a low-usage period. In some examples, the VNA / AI engine 350 can specify one or more thresholds to one or more of the pair of network devices that indicate that the WAN link is in a high-usage period, such as a specified threshold for the amount of data received and / or transmitted by one or more of the pair of network devices 110, and if the amount of data received and / or transmitted by one or more of the pair of network devices 110 is above the specified threshold, the VNA / AI engine 350 can determine that the WAN link is in a high-usage period.

[0114] In some examples, the VNA / AI engine 350 can communicate with one or more of the pair of network devices to schedule a speed test of the WAN link during a maintenance window of one or more of the pair of network devices. The maintenance window of a network device can be a time period immediately after the network device is upgraded and / or restarted, such that the network device does not receive and / or send traffic via the WAN link or receives and / or sends a very small amount of traffic. In this way, the VNA / AI engine 350 can communicate with one or more of the pair of network devices connected via the WAN link to determine the occurrence of a maintenance window of one or more of the pair of network devices and schedule a speed test of the WAN link during the maintenance window of one or more of the pair of network devices.

[0115] In some examples, the VNA / AI engine 350 can communicate with one or more of the pair of network devices to schedule speed tests for the WAN link at different times of the day, and the different times of the day can be based at least in part on the occurrence of events associated with the WAN link. For example, the VNA / AI engine 350 can schedule a speed test for the WAN link based on the occurrence of a network congestion event on the WAN link. A network congestion event can occur on the WAN link when the WAN link experiences a large amount of network traffic and becomes overloaded. For example, a network congestion event can occur on the WAN link when the network traffic on the WAN link consumes substantially all of the bandwidth of the WAN link, e.g., when the network traffic on the WAN link consumes 99% or more of the maximum bandwidth of the WAN link.

[0116] The VNA / AI engine 350 can detect the occurrence of a network congestion event on a WAN link by monitoring the performance of the WAN link, for example, by monitoring WAN link metrics (e.g., latency, jitter, and loss), application metrics (e.g., retransmission and round-trip time of TCP acknowledgments), metrics regarding correlation of degraded link performance with high bandwidth usage, metrics regarding long-term link stability, and / or quality of service of the WAN link (e.g., mean opinion score). For example, if the VNA / AI engine 350 determines that the latency, jitter, and / or loss of the WAN link suddenly exceeds a specified threshold within a specified time period, the VNA / AI engine 350 can determine that a network congestion event has occurred on the WAN link.

[0117] The VNA / AI engine 350 can determine the time of day that a network congestion event is likely to occur on the WAN link based on the occurrence of the network congestion event on the WAN link, and can schedule a speed test for the WAN link at a time of the day that is a specified amount of time (e.g., 1 hour) before the network congestion event is likely to occur. For example, if the VNA / AI engine 350 detects the occurrence of a network congestion event at 12:12 p.m. every day, the VNA / AI engine 350 can determine that the network congestion event is likely to occur at 12:12 p.m. every day. Therefore, the VNA / AI engine 350 can schedule a speed test for the WAN link at a time of the day that is a specified amount of time before the network congestion event is likely to occur. In the example where the VNA / AI engine 350 determines that the network congestion event is likely to occur at 12:12 p.m. every day, the VNA / AI engine 350 can schedule a speed test for the WAN link at 11:12 a.m. every day, i.e., one hour before the network congestion event is likely to occur.

[0118] Scheduling a speed test of the WAN link before a network congestion event may occur can enable the VNA / AI engine 350 to obtain one or more parameters related to the WAN link, which can enable the ML model 356 to learn the behavior of the WAN link before the network congestion event occurs, such as the available bandwidth of the WAN link before the network congestion event occurs. For example, the ML model 356 can use the parameters obtained by performing a speed test of the WAN link before the network congestion event occurs to determine whether the network congestion event is caused by WAN link degradation or other problems in the network 102.

[0119] In this manner, the VNA / AI engine 350 can schedule one or more of a pair of network devices 110 connected via a WAN link to obtain one or more parameters related to the WAN link based at least in part on a maintenance window of one or more of the pair of network devices, low usage time associated with the WAN link, high usage time associated with the WAN link, and / or a bandwidth of the WAN link measured at a given point in time exceeding a threshold.

[0120] As described above, the VNA / AI engine 350 may obtain one or more parameters related to the WAN link based on the speed test, for example, by communicating with one or more of a pair of network devices 110 connected via the WAN link to invoke an application, e.g., Figure 2, which is configured to cause one or more of the pair of network devices 110 to obtain one or more parameters. The one or more parameters related to the WAN link may include a measurement of the bandwidth of the WAN link during a high-usage period of the WAN link and / or during a low-usage period of the WAN link and / or a measurement of the interface bandwidth of one or more interfaces of one or more of the pair of network devices 110 connected to the WAN link, which may indicate the behavior of the WAN link during the high-usage period of the WAN link and / or during the low-usage period of the WAN link.

[0121] The one or more parameters related to the WAN link may include a measurement of the bandwidth of the WAN link during a maintenance window of one or more of the pair of network devices 110 and / or a measurement of the interface bandwidth of one or more interfaces of one or more of the pair of network devices 110 connected to the WAN link, which may indicate the behavior of the WAN link during a high usage period of the WAN link and / or a low usage period of the WAN link. The one or more parameters related to the WAN link may include a measurement of the bandwidth of the WAN link and / or a measurement of the interface bandwidth of one or more interfaces of one or more of the pair of network devices 110 connected to the WAN link at a specific time of day (e.g., a time of day before a network congestion event occurs on the WAN link), which may indicate that one or more problems with the WAN link may occur at the specific time of day.

[0122] The VNA / AI engine 350 may apply the ML model 356 to one or more parameters associated with a WAN link between a pair of network devices 110 in the network 102 to predict the maximum bandwidth of the WAN link. That is, the VNA / AI engine 350 may input one or more parameters associated with the WAN link obtained from a speed test performed on the WAN link into the ML model 356, and the ML model 356 may predict the maximum bandwidth of the WAN link based on the one or more parameters associated with the WAN link. The VNA / AI engine 350 may store the predicted maximum bandwidth of the WAN link in the database 315.

[0123] In some examples, NMS 300 may determine that a WAN link between a pair of network devices 110 is experiencing poor performance based on various metrics associated with the WAN link that NMS 300 may receive from, for example, the pair of network devices 110. Such metrics may include WAN link metrics (e.g., latency, jitter, and loss), application metrics (e.g., retransmission and round-trip time of TCP acknowledgments), metrics regarding correlation of degraded link performance with high bandwidth usage, metrics regarding long-term link stability, and / or quality of service (e.g., mean opinion score).

[0124] In some examples, the root cause analysis module 354 can use the predicted maximum bandwidth of the WAN link to detect a problem (e.g., a failure) with the WAN link at a given point in time and / or determine the root cause of the problem with the WAN link. For example, the root cause analysis module 354 can detect a performance problem with the WAN link by measuring the bandwidth of the WAN link at a given point in time and comparing the measured bandwidth of the WAN link with the predicted maximum bandwidth of the WAN link. The root cause analysis module 354 can measure the bandwidth of the WAN link connecting the pair of network devices 110 at a point in time based on one or more measurements of the amount of data sent by one of the network devices 110 for each communication session in a plurality of communication sessions for which the network device forwards network traffic, the amount of data received by the network device for each communication session in the plurality of communication sessions, the duration of each communication session in the plurality of communication sessions, the number of the plurality of communication sessions, the total bandwidth of an interface of the network device, and / or jitter, packet loss, or latency of network data associated with a communication session in the plurality of communication sessions. If root cause analysis module 354 determines that the measured bandwidth of the WAN link is less than the predicted maximum bandwidth of the WAN link by at least a predetermined amount, e.g., 90% less than the predicted maximum bandwidth of the WAN link, 95% less than the predicted maximum bandwidth of the WAN link, and so on, then root cause analysis module 354 may determine that there is a performance problem associated with the WAN link and may determine that a root cause of the poor performance of the WAN link is due to link exhaustion.

[0125] In response to the root cause analysis module 354 determining a root cause of the poor WAN link performance, the VNA / AI engine 350 may output a notification indicating the determined root cause of the poor WAN link performance. For example, if the root cause analysis module 354 may determine that link exhaustion of the WAN link is the root cause of the poor WAN link performance, the VNA / AI engine 350 may output a notification indicating that link exhaustion is the root cause of the poor WAN link performance.

[0126] In some examples, the VNA / AI engine 350 can determine one or more remedial actions to address poor performance of a WAN link. A pair of network devices 110 can be connected to each other via multiple WAN links on different interfaces of the pair of network devices 110. If the VNA / AI engine 350 determines that a WAN link connecting the pair of network devices 110 is link-depleted, the VNA / AI engine 350 can determine whether another WAN link connecting the pair of network devices 110 has available bandwidth to handle some network traffic of the link-depleted WAN link, and if so, can determine a remedial action of switching some network traffic from the link-depleted WAN link to another WAN link between the pair of network devices 110 that has available bandwidth to handle some network traffic of the link-depleted WAN link.

[0127] For example, the VNA / AI engine 350 may measure the bandwidth of each of the plurality of WAN links connecting the pair of network devices 110 at a given point in time, and may compare the measured bandwidth of each WAN link with the predicted maximum bandwidth of the corresponding WAN link. If the measured bandwidth of the WAN link is greater than a specified percentage of the predicted maximum bandwidth of the WAN link, for example, greater than 50%, 70%, etc., of the predicted maximum bandwidth of the WAN link, the VNA / AI engine 350 may determine that the WAN link is not link exhausted and has available bandwidth to accommodate the additional network traffic.

[0128] The VNA / AI engine 350 can determine a remedial action to address poor performance of a WAN link connecting a pair of network devices 110 based on comparing the measured bandwidth of each of a plurality of WAN links connecting the pair of network devices 110 with the corresponding predicted maximum bandwidth. For example, a first network device of the pair of network devices 110 can be connected to a first WAN link via a first interface, which in turn is connected to a second network device of the pair of network devices, and can be connected to a second WAN link via a second interface, which in turn is connected to the second network device. The VNA / AI engine 350 can determine that the first WAN link has a measured bandwidth of 2% of the predicted maximum bandwidth and is therefore link exhausted, and can determine that the second WAN link has a measured bandwidth of 80% of the predicted maximum bandwidth and is therefore not link exhausted. The VNA / AI engine 350 can therefore determine that the second WAN link has available bandwidth to handle some of the network traffic of the first WAN link and can determine a remedial action to avoid using the first WAN link to forward network traffic and / or switch network traffic from the first WAN link to the second WAN link to address the link exhaustion of the first WAN link.

[0129] In some examples, in response to determining that a remedial action is to switch network traffic from a first WAN link to a second WAN link to address link exhaustion of the first WAN link, the VNA / AI engine 350 can output a notification (e.g., to a network administrator of network 102) with a recommendation to stop forwarding network traffic using the first WAN link and / or switch some network traffic transmitted via the first WAN link from the first WAN link to the second WAN link. In some examples, in response to determining that a remedial action is to switch network traffic from a first WAN link to a second WAN link to address link exhaustion of the first WAN link, the VNA / AI engine 350 can automatically invoke the remedial action to switch some network traffic transmitted via the first WAN link from the first WAN link to the second WAN link, for example, by automatically switching one or more applications that are using the first WAN link to transmit data to use the second WAN link to transmit data.

[0130] Figure 4 is a block diagram illustrating an example implementation of NMS 130 in accordance with the techniques of this disclosure. Figure 4 is Figure 1A-Figure 1C described in the context of .

[0131] like Figure 4 As shown, the distributed computing system 478 as part of the NMS 130 can be used with Figure 1A The example of any of the network devices 110 shown, network device 410, communicates to perform a speed test of a WAN link between network device 410 and another network device, thereby obtaining one or more parameters of the WAN link and predicting the maximum bandwidth of the WAN link based on the obtained one or more parameters. An example of the distributed computing system 478 is an Apache Spark-based computing system capable of performing distributed processing on the obtained one or more parameters to predict the maximum bandwidth of the WAN link.

[0132] The distributed computing system 478 may include a WAN bandwidth module 480 that can be executed to predict the maximum bandwidth of the WAN link based on one or more parameters obtained related to the WAN link. The distributed computing system 478 may also include a speed test module 482 that can be executed to schedule a speed test of the WAN link to obtain one or more parameters of the WAN link. For example, the distributed computing system 478 may use an agent service 490 to communicate with the network device 410 to schedule and / or request the execution of a speed test of the WAN link according to the techniques described throughout this application.

[0133] The endpoint terminator 470 may establish a secure connection between the network device 410 and the NMS 130. The network device 410 may use the endpoint terminator 470 to stream parameters obtained by performing a speed test of the WAN link to a storage service 476 via a streaming platform 474 (e.g., Apache Kafka), which may be a cloud-based storage service, to store the obtained parameters for use by the WAN bandwidth module 480 to predict the maximum bandwidth of the WAN link based on the parameters related to the WAN link.

[0134] The distributed computing system 478 can store the predicted maximum bandwidth of the WAN link in a WAN bandwidth data store 484, which can be a data structure in memory, such as a remote dictionary server (Redis). The NMS 130 can use a protocol application programming interface (PAPI) 486 to access the predicted maximum bandwidth of the WAN link stored in the WAN bandwidth data store 484.

[0135] Figure 5 is a flow chart illustrating example operations performed by an example network management system. Figure 1A-Figure 1C 、 Figure 2 as well as Figure 3 Described Figure 5 .

[0136] like Figure 5 As shown, the processor(s) 306 of the network management system 300 may instruct a first network device (e.g., router 187A) to obtain one or more parameters associated with a wireless area network (WAN) link (e.g., logical path 189) between the first network device and a second network device (e.g., router 187B) (502). For example, the processor 306 may schedule the first network device to obtain the one or more parameters associated with the WAN link based at least in part on one or more of: a maintenance window for the first network device, a low usage period associated with the WAN link, a high usage period associated with the WAN link, or a bandwidth of the WAN link measured at a given point in time exceeding a predetermined threshold.

[0137] In some examples, the WAN link comprises a software-defined wide area network (SD-WAN) link. In some examples, the network management system 300 is physically separated from the first network device and the second network device. In some examples, to instruct the first network device to obtain one or more parameters related to the WAN link, the processor 306 can invoke an application (e.g., the speed test agent 238) executed by the first network device, the application being configured to cause the first network device to obtain the one or more parameters.

[0138] The processor 306 may apply the machine learning model 356 trained using the link parameters to one or more parameters associated with the WAN link to predict the bandwidth of the link, thereby predicting the maximum bandwidth of the WAN link (504). In some examples, the processor 306 may train the machine learning model using training data that includes a corresponding set of parameters and an associated maximum bandwidth for each link. In some examples, the corresponding set of parameters for the link includes: measured values ​​of the bandwidth of the link associated with different utilization ratios associated with the link; measured values ​​of the bandwidth of the interface of the corresponding network device associated with different utilization ratios associated with the link; measured values ​​of the bandwidth of the link associated with a maintenance window of the corresponding network device; and measured values ​​of the bandwidth of the link associated with different times of the day. The processor 306 may output an indication of the predicted maximum bandwidth of the WAN link (506).

[0139] In some examples, processor 306 may compare the bandwidth of the WAN link measured at a given point in time with a predicted maximum bandwidth of the WAN link, and may determine the occurrence of a WAN link failure in response to determining that the bandwidth of the WAN link measured at the given point in time is less than the predicted maximum bandwidth by a predetermined amount. In some examples, processor 306 may determine the bandwidth of the WAN link measured at the given point in time based at least in part on one or more of: an amount of data sent by the first network device for each of a plurality of communication sessions for which the first network device forwards network traffic; an amount of data received by the first network device for each of the plurality of communication sessions; a duration of each of the plurality of communication sessions; a number of the plurality of communication sessions; a total bandwidth of an interface of the first network device; and a measurement of jitter, packet loss, or delay of network data associated with a communication session in the plurality of communication sessions.

[0140] In some examples, processor 306 can output an indication of a WAN link failure based at least in part on determining the occurrence of a WAN link failure. In some examples, processor 306 can execute remedial measures to cause the first network device to avoid forwarding network traffic via the WAN link based at least in part on determining the occurrence of a WAN link failure.

[0141] The techniques described in this disclosure may be implemented, at least in part, in hardware, software, firmware, or any combination thereof. For example, various aspects of the described techniques may be implemented in one or more processors, including one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or any other equivalent integrated or discrete logic circuits, and any combination of these components. The term "processor" or "processing circuitry" may generally refer to any of the aforementioned logic circuits (alone or in combination with other logic circuits) or any other equivalent circuits. A control unit including hardware may also perform one or more techniques of the present disclosure.

[0142] Such hardware, software, and firmware may be implemented within the same device or within different devices to support the various operations and functions described in this disclosure. Furthermore, any described units, modules, or components may be implemented together or separately as discrete but interoperable logic devices. Describing different features as modules or units is intended to highlight different functional aspects and does not necessarily mean that these modules or units must be implemented by separate hardware or software components. Rather, the functions associated with one or more modules or units may be performed by separate hardware or software components, or integrated into common or separate hardware or software components.

[0143] The techniques described in this disclosure may also be embodied or encoded in a computer-readable medium (e.g., a computer-readable storage medium) containing instructions. The instructions embedded or encoded in the computer-readable storage medium may cause a programmable processor or other processor to perform the method, for example, when the instructions are executed. The computer-readable storage medium may include random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, hard disk, CD-ROM, floppy disk, cassette, magnetic media, optical media, or other computer-readable media.

[0144] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. A network management system, comprising: Memory; as well as One or more processors in communication with the memory and configured to: Instructing the first network device to obtain one or more parameters related to a wireless local area network (WAN) link between the first network device and the second network device; Applying a machine learning model trained using link parameters to the one or more parameters associated with the WAN link to predict a link bandwidth, thereby predicting a maximum bandwidth of the WAN link; as well as An indication of a predicted maximum bandwidth of the WAN link is output.

2. The network management system according to claim 1, wherein: The one or more processors are further configured to: comparing a bandwidth of the WAN link measured at a given point in time with the predicted maximum bandwidth of the WAN link; as well as In response to determining that the bandwidth of the WAN link measured at the given point in time is less than the predicted maximum bandwidth by a predetermined amount, an occurrence of a failure of the WAN link is determined.

3. The network management system according to claim 2, wherein: The one or more processors are further configured to determine the bandwidth of the WAN link measured at a given point in time based at least in part on one or more of: an amount of data sent by the first network device for each of a plurality of communication sessions for which the first network device forwards network traffic; an amount of data received by the first network device for each of the plurality of communication sessions; a duration of each of the plurality of communication sessions; a number of the plurality of communication sessions; The total bandwidth of the interface of the first network device; as well as One or more of measurements of jitter, packet loss, and delay of network data associated with a communication session in the plurality of communication sessions.

4. The network management system according to claim 2, wherein: The one or more processors are configured to output an indication of the WAN link failure based at least in part on determining the occurrence of the WAN link failure.

5. The network management system according to claim 2, wherein: The one or more processors are configured to, based at least in part on determining the occurrence of the WAN link failure, execute remedial action to cause the first network device to refrain from forwarding network traffic via the WAN link.

6. The network management system according to any one of claims 1 to 5, wherein: The one or more processors are further configured to schedule the first network device to obtain the one or more parameters associated with the WAN link based at least in part on one or more of: a maintenance window for the first network device; low usage times associated with the WAN link; high usage times associated with the WAN link; as well as The bandwidth of the WAN link measured at a given point in time exceeds a predetermined threshold.

7. The network management system according to claim 1, wherein: The one or more processors are further configured to train the machine learning model using training data, the training data comprising a corresponding set of parameters and an associated maximum bandwidth for each link.

8. The network management system according to claim 7, wherein: The corresponding set of parameters for the link includes one or more of the following: measures of bandwidth of the link associated with different usage ratios associated with the link; measures of bandwidth of interfaces of respective network devices associated with different utilization ratios associated with the links; a measurement of the bandwidth of the link associated with a maintenance window of the corresponding network device; as well as Measures of the bandwidth of the link associated with different times of day.

9. The network management system according to claim 1, wherein: The WAN link includes a software-defined wide area network (SD-WAN) link.

10. The network management system according to claim 1, in, The network management system is physically separated from the first network device and the second network device, and In order to instruct the first network device to obtain the one or more parameters related to the WAN link, the one or more processors are configured to call an application executed by the first network device, and the application is configured to enable the first network device to obtain the one or more parameters.

11. A computer networking method, comprising: Instructing the first network device to obtain one or more parameters related to a wireless local area network (WAN) link between the first network device and a second network device by one or more processors of the network management system; applying, by the one or more processors, a machine learning model trained using link parameters to the one or more parameters associated with the WAN link to predict a link bandwidth, thereby predicting a maximum bandwidth of the WAN link; as well as An indication of a predicted maximum bandwidth of the WAN link is outputted by the one or more processors.

12. The computer networking method according to claim 11, further comprising: comparing, by the one or more processors, a bandwidth of the WAN link measured at a given point in time with the predicted maximum bandwidth of the WAN link; as well as An occurrence of a failure of the WAN link is determined, by the one or more processors, in response to determining that the bandwidth of the WAN link measured at the given point in time is less than the predicted maximum bandwidth by a predetermined amount.

13. The computer networking method according to claim 12, wherein: Determining the bandwidth of the WAN link measured at a given point in time further comprises: The bandwidth of the WAN link measured at a given point in time is determined by the first network device based at least in part on one or more of: an amount of data sent by the first network device for each of a plurality of communication sessions for which the first network device forwards network traffic; an amount of data received by the first network device for each of the plurality of communication sessions; a duration of each of the plurality of communication sessions; a number of the plurality of communication sessions; The total bandwidth of the interface of the first network device; and One or more of measurements of jitter, packet loss, and delay of network data associated with a communication session of the plurality of communication sessions.

14. The computer networking method according to claim 12, further comprising: An indication of the WAN link failure is outputted by the one or more processors based at least in part on determining the occurrence of the WAN link failure.

15. The computer networking method according to claim 12, further comprising: Remedial action is performed, by the one or more processors, based at least in part on determining the occurrence of the WAN link failure to cause the first network device to refrain from forwarding network traffic via the WAN link.

16. The computer networking method according to any one of claims 11 to 15, further comprising: Scheduling, by the one or more processors, the first network device to obtain the one or more parameters associated with the WAN link based at least in part on one or more of: a maintenance window for the first network device; low usage times associated with the WAN link; a high usage period associated with the WAN link; or The bandwidth of the WAN link measured at a given point in time exceeds a predetermined threshold.

17. The computer networking method according to claim 11, further comprising: The machine learning model is trained using training data including a corresponding set of parameters and an associated maximum bandwidth for each link.

18. The computer networking method according to claim 17, wherein: The corresponding set of parameters for the link includes one or more of the following: measures of bandwidth of the link associated with different usage ratios associated with the link; measures of bandwidth of interfaces of respective network devices associated with different utilization ratios associated with the links; a measurement of the bandwidth of the link associated with a maintenance window of the corresponding network device; or Measures of the bandwidth of the link associated with different times of day.

19. The computer networking method according to claim 11, wherein: The WAN link includes a software-defined wide area network (SD-WAN) link.

20. A computer-readable storage medium comprising instructions which, when executed, cause one or more processors of a network management system to: Instructing the first network device to obtain one or more parameters related to a wireless local area network (WAN) link between the first network device and the second network device; Applying a machine learning model trained using link parameters to the one or more parameters associated with the WAN link to predict a link bandwidth, thereby predicting a maximum bandwidth of the WAN link; and An indication of a predicted maximum bandwidth of the WAN link is output.

Images