Network diagnostics for controlling paths between a partner network and a wan

Abstract

This application relates to communication between a partner network and a wide area network (WAN) via the Internet. The WAN advertises unicast border gateway protocol (BGP) address prefixes for a plurality of front-end devices in the WAN. A proxy in the partner network measures a plurality of paths to a service within the WAN. Each of the plurality of paths is associated with one of the plurality of front-end devices and a corresponding unicast BGP address prefix. The proxy provides measurements of the plurality of paths to the WAN. The WAN selects a path within the WAN for the service. The proxy receives a routing rule specifying a unicast address prefix for a selected device of the plurality of front-end devices of the WAN. The proxy forwards data packets for the service to the corresponding border gateway protocol address prefix of the selected device via the Internet.

Description

Background Technology

[0001] Wide area networks (WANs) can include computing resources distributed across geographical regions and connected via communication links such as fiber optic cables. The size of WANs can vary greatly, from small city networks to global networks. For example, a WAN might connect multiple offices of a company, customers of a regional telecommunications operator, or a global enterprise. The computing resources and connections within a WAN can be owned and controlled by the WAN operator.

[0002] A partner network can obtain services from a WAN. For example, a partner network may be an enterprise network, and the WAN may provide managed services to users of the enterprise network. The partner network can connect to the WAN via the Internet. The path between the partner network and the WAN can include one or more Internet Service Providers (ISPs). Typically, services over the Internet can be routed based on the lowest cost. Therefore, neither the partner network nor the WAN has significant control over routing decisions between them.

[0003] One attempt to give networks greater control over routing decisions is Software-Defined Wide Area Networking (SD-WAN). SD-WAN can control routing decisions within an enterprise network. SD-WAN devices, often referred to as SD-WAN edges, are physical or virtual network functions placed in an organization's branch / regional / central office locations, data centers, and public or private cloud platforms. SD-WAN edges can perform traffic classification and forwarding based on the availability of routes to other SD-WAN edges. However, in the context of cloud services hosted by a WAN operator, SD-WAN may lack the information or capability to select routes over the Internet to the hosted cloud service WAN. Summary of the Invention

[0004] The following is a simplified summary of one or more aspects to provide a basic understanding of these aspects. This invention is not a broad overview of all anticipated aspects, nor is it intended to identify key or essential elements of all aspects, nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that follows.

[0005] In this example, an apparatus for managing connectivity between a partner network and a wide area network (WAN) is provided. The apparatus includes a memory storing one or more instructions for managing the WAN and at least one processor coupled to the memory and configured to execute the instructions. The at least one processor is configured to advertise multiple unicast border gateway protocol address prefixes for multiple front-end devices of the WAN. The at least one processor is configured to select a path within the WAN for services for users of the partner network, the path including a selected device among the multiple front-end devices. The at least one processor is configured to export routing rules to an agent within the partner network, the routing rules specifying the unicast address prefixes for the selected devices. The at least one processor is configured to configure the selected devices to receive packets for services for the partner network.

[0006] In another example, a method is provided for communication between a partner network and a service hosted in a WAN connected to the partner network via the Internet. The method includes measuring multiple paths to the service within the WAN, each path associated with one of a plurality of front-end devices in the WAN, the plurality of front-end devices being associated with a corresponding unicast border gateway protocol address prefix. The method includes providing the WAN with measurements of the multiple paths to the service. The method includes receiving a routing rule specifying the unicast address prefix of a selected device among the plurality of front-end devices in the WAN. The method includes forwarding data packets for the service via the Internet to the corresponding border gateway protocol address prefix of the selected device.

[0007] In some implementations, each unicast border gateway protocol address prefix in the corresponding unicast border gateway protocol address prefix identifies the point of presence (POP) or peer-to-peer connection between the WAN and the Internet service provider (ISP).

[0008] In some implementations, the receive routing rules occur at the software-defined wide area network (SD) device within the partner network.

[0009] In some implementations, the receive routing rules occur at the application controlled by the WAN.

[0010] In some implementations, forwarding service-specific data packets involves encapsulating service-specific packets within packets that use a unicast address prefix for the selected device, based on routing rules.

[0011] In some implementations, forwarding data packets for a service involves rewriting the header of the packet for the service based on routing rules and the port corresponding to the original address of the service, using a unicast address prefix for the selected device.

[0012] In some implementations, measurements include latency, delay, throughput, or the devices traversed.

[0013] In some implementations, the measurement includes a trajectory route that indicates one or more networks between the partner network and the WAN.

[0014] In some implementations, measuring multiple paths to services within the WAN includes: measuring a portion of each path to a corresponding front-end device among multiple front-end devices in the WAN via the respective unicast border gateway protocol address prefix.

[0015] To achieve the foregoing and related objectives, one or more aspects include the features fully described and specifically pointed out in the claims. The following description and accompanying drawings set forth certain illustrative features of one or more aspects in detail. However, these features indicate only a few of the various ways in which the principles of each aspect can be employed, and this description is intended to include all such aspects and their equivalents. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating an example of an architecture for connecting a partner network to a wide area network (WAN), based on the aspects described in this article.

[0017] Figure 2 This is a diagram illustrating an example architecture for connecting a partner network to a WAN via a selected path, based on the aspects described in this article.

[0018] Figure 3 This is a diagram illustrating example components of a routing agent and a routing controller, and the communication between them, based on the aspects described in this article.

[0019] Figure 4 This is a schematic diagram illustrating an example of a device used to route services from a partner network to a WAN, based on the aspects described herein.

[0020] Figure 5 This is a flowchart illustrating an example of a method for routing services from a partner network to a WAN, based on the aspects described in this article.

[0021] Figure 6 This is a schematic diagram of an example of a device used to perform the functions of the routing agent described herein, based on the aspects described herein.

[0022] Figure 7 This is a schematic diagram of an example of a device used to control communication between a partner network and a WAN, based on the aspects described herein.

[0023] Figure 8 This is a flowchart illustrating an example of a method for controlling communication between a partner network and a WAN, based on the aspects described herein.

[0024] Figure 9 This is a schematic diagram of an example of a device for performing the routing controller functions described herein, based on the aspects described herein. Detailed Implementation

[0025] The specific embodiments described below with reference to the accompanying drawings are intended as descriptions of various configurations and are not intended to represent the only configurations in which the concepts described herein can be practiced. The specific embodiments include particular details for the purpose of providing a thorough understanding of the various concepts. However, it will be apparent to those skilled in the art that these concepts can be practiced without these particular details. In some cases, well-known components are shown in block diagram form to avoid obscuring these concepts.

[0026] This disclosure describes various examples related to path selection between a partner network and a wide area network (WAN). In one aspect, a WAN includes multiple front-end devices connected to an Internet Service Provider (ISP) at peer locations such as points of presence (POPs). Typically, traffic for services hosted on the WAN can enter the WAN at any front-end device. Traditionally, front-end devices announce their presence to the ISP using Border Gateway Protocol (BGP) anycast advertisements. Such advertisements associate the same Internet Protocol (IP) address and prefix with each front-end device. Therefore, traffic for services hosted on the WAN can be forwarded by the ISP to any front-end device.

[0027] In one respect, communication attributes (e.g., Quality of Service (QoS) or Quality of Experience (QoE)) between a partner network and services hosted in a WAN can be improved by selecting at least a portion of the path between the partner network and the services. For example, services carried by different ISPs may have different characteristics in terms of latency, jitter, etc. Furthermore, the path within the WAN from the front-end device to the service may have different characteristics. However, because inbound traffic may traditionally reach any front-end device, the WAN may have limited options for routing traffic to services.

[0028] In one aspect, this disclosure provides path selection between a partner network and the WAN by advertising different BGP address prefixes for different front-end devices of the WAN. For example, the WAN can advertise unicast BGP address prefixes for multiple front-end devices. Furthermore, the WAN can implement a proxy within the partner network (e.g., at an SD-WAN device) that measures paths involving different front-end devices of the WAN. Thus, the WAN can identify paths from the partner network via the Internet to services hosted within the WAN with better characteristics. The WAN can then control the proxy to route traffic to the service along the selected path. For example, the proxy can tunnel traffic to the unicast BGP address prefix associated with the selected path. Therefore, the WAN can improve connectivity between the partner network and the WAN via the Internet by selecting specific paths for traffic associated with services.

[0029] Turn now Figure 1-9 Examples are depicted with reference to one or more components and one or more methods that can perform the actions or operations described herein, wherein the components and / or actions / operations in the dashed lines may be optional. Although Figure 4 The operations described below are presented in a specific order and / or performed by example components, but in some examples, the order of the actions and the components performing the actions may vary depending on the implementation. Furthermore, in some examples, one or more of the actions, functions, and / or described components may be performed by a specific programming processor, a processor executing specific programming software or computer-readable media, or by any other combination of hardware and / or software components capable of performing the described actions or functions.

[0030] Figure 1 This is a conceptual diagram 100 illustrating an example of an architecture for connectivity between a partner network 110 and a wide area network (WAN) 140 via the Internet 120. The WAN 140 may host services 146 for the partner network 110.

[0031] Partner network 110 may include any computer network that can be connected to WAN 140 via Internet 120. For example, partner network may include an enterprise network, which may itself be a WAN connecting multiple locations of the enterprise. As another example, partner network 110 may include a radio access network (RAN). For example, partner network 110 may include user equipment 104 wirelessly connected to base station 102. Partner network 110 may include user plane functions (UPF) 106 that handle user traffic from the core network. In some implementations, service 146 may include core network functions.

[0032] In some cases, the partner network may utilize the Software-Defined Wide Area Network (SD-WAN) device 112. For example, the SD-WAN device 112 may be an SD-WAN edge device that controls routing to other SD-WAN edge devices and to the Internet 120 and / or WAN 140. In some implementations, the SD-WAN device may include software running on network devices such as routers or servers. For example, the SD-WAN device 112 may be provided by a third party and run on devices in the partner network 110.

[0033] Internet 120 may be a network of Internet Service Providers (ISPs) 122 connected according to the Internet Protocol. For simplicity, a first ISP 122a and a second ISP 122b are shown, but it should be understood that Internet 120 may include multiple ISPs directly or indirectly connected. ISPs may also be referred to as Autonomous Systems (AS) and associated with an Autonomous System Number (ASN), which can be used for routing according to the Border Gateway Protocol (BGP).

[0034] WAN 140 may include computing resources distributed across geographical areas and connected via communication links such as fiber optic cables. For example, WAN 140 may include front-end devices 142 and data centers 144. For example, front-end devices 142 may be referred to as edge devices and may include routers and / or servers. Front-end devices 142 may be located at a point of presence (POP) and have peering connections to one or more ISPs 122. Peering connections may be associated with a BGP IP address prefix 143. For example, each front-end device 142 may use a BGP anycast advertisement to establish the same IP address prefix (1.1.1.1 / 24). Therefore, ISP 122 can forward traffic to WAN 140 through peering connections. Data centers 144 may include computing resources (e.g., servers) within WAN 140. For example, in a cloud computing scenario, WAN 140 may host services 146 (e.g., applications) at one or more locations within data centers 144. Data centers 144 may also include routers configured to forward traffic to front-end devices 142 and / or other data centers 144. In some implementations, service 146 may be a transport service associated with the egress point of WAN 140, which may be another front-end device 142.

[0035] like Figure 1As shown, services for service 146 can take different paths from partner network 110 to WAN 140. For example, a first path 150 can travel from SD-WAN device 112 to first ISP 122a, to front-end device 142a, and to data centers 144a, 144b, and 144c. As another example, a second path 152 can travel from SD-WAN device 112 to second ISP 122b, to front-end device 142b, and to data center 144c. In some cases, SD-WAN device 112 may provide a limited selection between paths 150 and 152, but this selection may only apply to the selection of the initial ISP 122, as the initial ISP 122 may choose to forward the service to another ISP instead of directly to WAN 140. Furthermore, as shown, path selection by WAN 140 is limited because the service may reach any front-end device 142.

[0036] Figure 2 This is a conceptual diagram 200 illustrating an example of an architecture for connectivity between partner network 210 and WAN 240. This architecture uses a routing agent 230 within partner network 210 to route traffic via Internet 220 to selected front-end devices on WAN 240. WAN 240 includes a routing controller 250 that communicates with the routing agent 230 and provides routing policies to the routing controller 250.

[0037] Partner network 210 may be similar to partner network 110 and includes user equipment 104. In the RAN case, partner network 210 may include base station 102 and UPF 106. Partner network 210 may include routing agent 230. In some implementations, routing agent 230 may be implemented on SD-WAN device 112. In some implementations, routing agent 230 may be implemented on other devices in partner network 210, including, for example, user equipment 104.

[0038] WAN 240 may be similar to WAN 140 and includes front-end device 142 and data center 144. In one aspect, one or more of the front-end devices 142 may advertise a unicast BGP address prefix for the front-end device 142. For example, front-end device 142a may send advertisement 242 to ISP 122a. Advertisement 242 may advertise a BGP address prefix 243a (e.g., 1.1.2 / 24) for front-end device 142a. Because advertisement 242 is a BGP unicast advertisement, ISP 122a may forward traffic for the specified address prefix 243a to the specific front-end device 142a. ISP 122a may propagate advertisement 242 to establish paths to front-end device 142a through other ISPs 122. Similarly, front-end device 142b may send advertisement 244 to ISP 122b advertising a BGP address prefix 243b (e.g., 1.1.3 / 24) for front-end device 142b. In one aspect, the routing controller 250 can control the front-end devices 142 to advertise multiple unicast BGP address prefixes for multiple front-end devices 142 on the WAN 240. Therefore, the routing controller 250 can establish paths through the Internet 220 to specific front-end devices 142.

[0039] In one aspect, front-end device 142 can also advertise anycast BGP prefix 143 shared by front-end devices 142. Anycast BGP prefix 143 can be unspecified compared to unicast BGP prefix 243 (e.g., 1.1 / 16). Depending on BGP, ISP 122 can select a more specific route for unicast BGP prefix 243 if available. Anycast BGP prefix 143 can gracefully handle certain types of failures. For example, if multiple front-end devices are associated with the same anycast BGP prefix 143, and one of the front-end devices associated with unicast BGP prefix 243 becomes unavailable, ISP 122 can select a different route to anycast BGP prefix 143. In one aspect, unicast BGP address prefix 243 may be limited by availability and / or cost. Route controller 250 can select which front-end devices 142 are associated with the unicast BGP address prefix. For example, routing controller 250 can analyze the network topology of WAN 240 to determine front-end devices 142 that may be associated with a desired (e.g., relatively low latency or greater bandwidth) path to service 146 within WAN 240. In some implementations, unicast BGP address prefixes can be reused by front-end devices in geographically isolated areas of WAN 140. For example, if front-end device 142c is located in a different region or country, making it unlikely that traffic from ISP 122b will be routed to front-end device 142c, front-end device 142c can also advertise the 1.1.3 / 24 unicast BGP address prefix to establish paths in its respective area.

[0040] In one respect, the advertisement of unicast BGP address prefixes can establish paths 260 and 262. Figure 1 Unlike paths 150 and 152, routes 250 and / or route agents 230 can have greater control over paths 260 and 262. For example, route agent 230 can tunnel traffic for service 146 to the corresponding front-end device 142 associated with the selected path. Although ISP 122 can be an autonomous system and choose different routes through Internet 220, the use of unicast BGP address prefixes ensures that traffic reaches a specific entry point on WAN 240. Furthermore, the distinction between paths 260 and 262 allows route agent 230 to measure paths 260 and 262. For example, route agent 230 can send ping packets to the service along each available path to determine the latency associated with each path. Again, although ISP 122 can make different routing decisions, such decisions can be relatively static and produce consistent path measurements.

[0041] Routing controller 250 can select a path (e.g., path 260, 262) within WAN 240 for service 146 of users of partner network 210. Paths 260, 262 include selected devices among multiple front-end devices 142. For example, routing controller 250 can select a path based on internal routing preferences of WAN 240. For example, routing controller 250 can select a path with the lowest latency or the maximum available bandwidth. In some implementations, routing controller 250 can select a path based on measurements 232 provided by routing agent 230. In some implementations, routing agent 230 can provide measurements 232 of paths 260, 262 to the corresponding front-end devices 142. Such measurements 232 can represent portions of WAN 240 in which direct control over routing is lacking. Routing controller 250 can perform measurements of various paths within WAN 240. In some cases, routing controller 250 can add measurements 232 from the front-end device 142 to the measurements 232 of the path portion outside WAN 240. In other implementations, routing agent 230 can measure total paths 260 and 262. Therefore, routing controller 250 can dynamically select the path between partner network 210 and service 146.

[0042] Routing controller 250 can control routing agent 230 to use the selected path for the service 146. For example, routing controller 250 can export routing rule 252 to routing agent 230. Routing rule 252 can specify a unicast address prefix 243 for the selected front-end device 142. Routing agent 230 can forward the service to the unicast address prefix based on the routing rule. For example, routing agent 230 can encapsulate service packets within packets with the unicast address prefix for the selected front-end device 142 based on the routing rule. Then, front-end device 142 can decapsulate service packets from packets addressed to the unicast address prefix for the selected device based on the routing rule. As another example, routing agent 230 can rewrite the header of service packets using the unicast address prefix for the selected device based on the routing rule. In some implementations, the header may include the port corresponding to the original address of the service. Then, front-end device 142 can forward data packets addressed to the unicast address prefix for the selected device 142 to service 146 based on routing rules and the original address within WAN 240. Routing controller 250 can configure the routing table within WAN 240 so that front-end device 142 and other devices forward packets to service 146 along selected paths 260, 262. Therefore, routing controller 250 may have increased control over packet paths for the service, thereby improving system operation. For example, routing controller 250 can select paths 260, 262 to meet the SLA for service 146.

[0043] Figure 3 Figure 300 shows example components of routing agent 230 and routing controller 250 and the communication between them. Routing agent 230 may include measurement component 310, control component 312, forwarding component 314, and monitoring component 316. Routing controller 250 may include advertising component 350, path selection component 352, external rules component 354, and internal routing component 356.

[0044] Measurement component 310 can be configured to measure multiple paths to services within WAN 240. Each of the multiple paths can be associated with one of multiple front-end devices 142 of WAN 240, which are associated with a corresponding unicast border gateway protocol address prefix 243. In one aspect, measurement component 310 may receive a list of unicast border gateway protocol address prefixes 243 from routing controller 250. In some implementations where routing agent 230 connects to two or more ISPs 122, multiple available paths to each front-end device 142 may exist. For example, the path to edge device 142a using unicast address prefix 243a and the first ISP 122a may be different from the path to edge device 142a and the second ISP 122b using unicast address prefix 243a. Measurement component 310 can generate a measurement 232 for each potential path. Measurement 232 may include latency, delay, throughput, and / or the devices traversed. In some implementations, measurement 232 may include a trace route indicating one or more networks (e.g., ISP 122) between partner network 210 and WAN 240.

[0045] Control unit 312 can be configured to communicate with routing controller 250. For example, control unit 312 can establish a session with routing controller 250. In one aspect, the session can utilize anycast BGP address prefixes for reliability. Control unit 312 can provide measurement 232 to routing controller 250 via the session. Control unit 312 can receive routing rules 252 from routing controller 250 via the session.

[0046] Forwarding component 314 can be configured to forward data packets for service 146 via Internet 220 to the appropriate Border Gateway Protocol (BGP) address prefix 243 of the selected device 142. For example, forwarding component 314 can be a router or a software routing stack. Forwarding component 314 can identify packets for service 146. For example, forwarding component 314 can identify packets based on the five-tuple of each packet. Forwarding component 314 can tunnel packets to BGP address prefix 243. For example, forwarding component 314 can use encapsulation and / or header rewriting. In some implementations, forwarding component 314 can select ISP 122 to forward packets. The selected ISP can be based on routing rule 252. For example, routing controller 250 can determine ISP 122 on the selected paths 260, 262 based on measurement 232.

[0047] Monitoring component 316 can monitor the operation of routing agent 230. For example, the monitoring component can collect statistics about services forwarded according to routing rules. In some implementations, monitoring component 316 can generate an operator dashboard viewable by the partner network 210. For example, monitoring component 316 may include a user interface that allows network operators to view statistics. In some implementations, network operators can configure routing agent 230. For example, monitoring component 316 can allow network operators to select which services 146 receive routing rules.

[0048] The announcement component 350 can be configured to announce multiple unicast border gateway protocol address prefixes 243 for multiple front-end devices 142 of WAN 240. For example, the announcement component 350 can control multiple front-end devices 142 to send announcements 242 to a connected ISP 122. In some implementations, the announcement component 350 can select multiple front-end devices 142 based on the network topology of WAN 240. For example, the announcement component 350 can allocate a limited number of unique unicast border gateway protocol address prefixes 243 among a larger number of front-end devices 142. In some implementations, the announcement component 350 can select geographically distributed front-end devices. In some implementations, the announcement component 350 can select front-end devices associated with different paths to data center 144 or service 146.

[0049] Path selection component 352 can be configured to select paths 260, 262 within WAN 240 for service 146 of users of partner network 210. Paths 260, 262 may include selected devices from a plurality of front-end devices 142. For example, in one implementation, path selection component 352 may select the optimal path from any front-end device 142 associated with unicast BGP address prefix 143 to data center 144 of hosting service 146. For example, the optimal path may be the path with the lowest cost, the path with the least load, the path with the most bandwidth, the path with the best performance (e.g., latency and jitter), or the best combination of these metrics. In other implementations, path selection component 352 may receive measurements 232 from routing agent 230 and select paths based on measurements 232. The use of measurements 232 may advantageously take into account portions of paths 260, 262 on Internet 220 that may be responsible for significant latency. In one aspect, the path selection component 352 can be configured to select the optimal combination of the external portion of the path from the routing agent 230 to the front-end device 142 and the internal portion of the path from the front-end device 142 to the data center 144 or the hosting service 146.

[0050] External rule component 354 can be configured to export routing rule 252 to routing agent 230 within partner network 210. Routing rule 252 can specify at least one unicast address prefix 243 for the selected device. Therefore, routing agent 230 can route traffic for service 146 to the selected device. In some implementations, routing rule 252 can specify specific tunneling settings or ISPs. For example, routing rule 252 can specify encapsulation, header rewriting, or port forwarding settings.

[0051] The internal routing component 356 can be configured to control the selected front-end device 142 to receive data packets for service 146 of the partner network 210 at the selected device. For example, the internal routing component 356 can be configured to perform any packet processing (e.g., decapsulation or header rewriting) on ​​arriving data packets. The internal routing component 356 can configure routing tables at the front-end device 142 and each other device along the selected path to forward data packets to service 146.

[0052] Figure 4 This is a schematic diagram of an example device 400 (e.g., a computing device) for communication between a partner network and a service hosted in a WAN connected to the partner network via the Internet. Device 400 can be implemented as one or more computing devices in partner network 210. For example, device 400 can be implemented as SD-WAN device 112. Multiple user devices 104 can be configured to forward data packets to device 400. For example, user devices 104 can be connected to device 400 via a local area network (LAN) or a virtual private network (VPN). In some implementations where partner network 110 is a RAN, user devices 104 can be connected via a UPF 106 according to radio network specifications. In some implementations, UPF 106 can execute a software stack (e.g., according to radio network specifications) under the control of WAN 240. In some implementations, device 400 can be user device 104.

[0053] In one example, device 400 may include processor 402 and / or memory 404 configured to execute or store instructions or other parameters related to providing operating system 406, which may execute one or more applications or processes, such as, but not limited to, a routing agent 230 for forwarding data packets along a selected path to a WAN. For example, processor 402 and memory 404 may be discrete components coupled via bus communication (e.g., on a motherboard or other part of a computing device, on an integrated circuit, such as a system-on-a-chip (SoC)), components integrated internally to each other (e.g., processor 402 may include memory 404 as an on-board component), and / or similar components. Memory 404 may store instructions, parameters, data structures, etc., for use / execution by processor 402 to perform the functions described herein.

[0054] In one example, routing agent 230 may include a measurement component 310, a control component 312, and a forwarding component 314. Routing agent 230 may optionally include a monitoring component 316.

[0055] Figure 5 This is a flowchart illustrating an example of a method 500 for forwarding packets to WAN 240 based on a path selected by WAN 240. For example, method 500 may be performed by device 400 and / or one or more of its components to measure potential paths and route data packets along the selected path to a selected front-end device 142 of WAN 240.

[0056] In block 510, method 500 includes measuring multiple paths to services within the WAN, each of the multiple paths being associated with one of a plurality of front-end devices in the WAN, which are associated with corresponding unicast BGP address prefixes. In one example, routing agent 230 and / or measurement component 310, for example, in conjunction with processor 402, memory 404, and operating system 406, can measure multiple paths 260, 262 to service 146 within WAN 240. Each of the multiple paths 260, 262 is associated with one of a plurality of front-end devices 142a, 142b in WAN 240, which are associated with corresponding unicast BGP address prefixes 143a, 143b. In some implementations, the measurement includes latency, delay, throughput, or traversed devices. In some implementations, the measurement includes tracing routes indicating one or more networks between a partner network and the WAN. In some implementations, within subblock 512, block 510 can measure portions of each path 260, 262 to a corresponding one of the multiple front-end devices 142a, 142b of the WAN via the corresponding unicast border gateway protocol address prefixes 143a, 143b. That is, measurement component 310 can measure portions of paths 260, 262 outside the WAN 240. In such implementations, the measurement may not depend on internal routing within the WAN 240. The WAN 240 can determine the portion of the path within the WAN 240 based on internal measurements, network topology, and / or cost. In some other implementations, block 510 may include measuring the complete path between routing agent 230 and service 146. Furthermore, measurement component 310 can continue performing measurements after a path has been selected, for example, to determine whether to update the path.

[0057] In block 520, method 500 includes providing measurements of multiple paths to the service to the WAN. In one example, control unit 312 and / or routing agent 230, for example, in conjunction with processor 402, memory 404, and operating system 406, may provide measurements 232 of multiple paths 260, 262 to service 146 to WAN 240. For example, control unit 312 may communicate measurements 232 to routing controller 250. In some implementations, control unit 312 may provide measurements in response to a request from routing controller 250.

[0058] In block 530, method 500 includes receiving a routing rule that specifies a unicast address prefix for a selected device among a plurality of front-end devices in the WAN. In one example, control unit 312 and / or routing agent 230, for example, in conjunction with processor 402, memory 404, and operating system 406, may receive routing rule 252 that specifies a unicast address prefix 243 (e.g., prefix 243a) for a selected device (e.g., front-end device 142a) among a plurality of front-end devices 142 in the WAN 240.

[0059] In block 540, method 500 includes forwarding data packets for the service via the Internet to the appropriate Border Gateway Protocol (BGP) address prefix of the selected device. In one example, forwarding component 314 and / or routing agent 230, such as in combination with processor 402, memory 404, and operating system 406, can forward data packets 320 for service 146 via Internet 220 to the appropriate BGP address prefix 243a of the selected device 142a. For example, forwarding component 314 can establish a tunnel to the selected device 142a via Internet 220. In some implementations, in subblock 542, block 540 may include encapsulating service-specific packets within packets with a unicast address prefix for the selected device based on routing rules. In some implementations, in subblock 544, block 540 may include rewriting the header of service-specific packets using the unicast address prefix for the selected device based on routing rules and ports corresponding to the original addresses for the service. For example, the tunneling technique may depend on the capabilities and / or configuration of the selected front-end device 142.

[0060] Figure 6 Examples of device 600 are shown, including, for example Figure 4 Additional optional component details are shown. In one aspect, device 600 may include processor 602, which may be similar to processor 402, for performing processing functions associated with one or more components and functions described herein. Processor 602 may include a single processor or a set of multi-core processors. Furthermore, processor 602 may be implemented as an integrated processing system and / or a distributed processing system.

[0061] Device 600 may further include memory 604, which may be similar to memory 404, such as for storing a local version of an operating system (or its components) and / or applications executed by processor 602, such as routing agent 230, measurement unit 310, control unit 312, forwarding unit 314, monitoring unit 316, etc. Memory 604 may include computer-available memory types such as random access memory (RAM), read-only memory (ROM), magnetic tape, magnetic disk, optical disk, volatile memory, non-volatile memory, and any combination thereof.

[0062] Furthermore, device 600 may include a communication component 606, which provides, for example, the establishment and maintenance of communication with one or more other devices, parties, entities, etc., using hardware, software, and services as described herein. Communication component 606 may carry communication between components on device 600, and between device 600 and external devices, such as devices located across a communication network and / or devices serially or locally connected to device 600. For example, communication component 606 may include one or more buses and may further include transmit chain components and receive chain components, respectively associated with a wireless or wired transmitter and receiver operable for interconnecting with external devices.

[0063] Furthermore, device 600 may include data storage 608, which may be any suitable combination of hardware and / or software, providing massive storage for information, databases, and programs used in conjunction with the aspects described herein. For example, data storage 608 may be, or may include, a data repository for operating systems (or components thereof) not currently executed by processor 602, applications, related parameters, etc. Additionally, data storage 608 may be a data repository for routing agent 230.

[0064] Device 600 may optionally include a user interface component 610 operable to receive input from a user of device 600 and further operable to generate output for presentation to the user. User interface component 610 may include one or more input devices, including but not limited to a keyboard, numeric keypad, mouse, touch-sensitive display, navigation keys, function keys, microphone, voice recognition component, gesture recognition component, depth sensor, gaze tracking sensor, switch / button, any other mechanism capable of receiving input from the user, or any combination thereof. Furthermore, user interface component 610 may include one or more output devices, including but not limited to a display, speaker, haptic feedback mechanism, printer, any other mechanism capable of presenting output to the user, or any combination thereof.

[0065] The device 600 may also include a routing agent 230 for communicating between a partner network and services hosted in a WAN connected to the partner network via the Internet, a measurement unit 310 for measuring multiple paths to services within the WAN, a control unit 312 for providing measurements of multiple paths to services to the WAN and receiving routing rules, and a forwarding unit 314 for forwarding data packets for services to appropriate border gateway protocol address prefixes of selected devices via the Internet.

[0066] Figure 7 This is a schematic diagram of an example device 700 (e.g., a computing device) used to manage connectivity between a partner network and a WAN. Device 700 can be implemented as one or more computing devices in WAN 240. For example, device 700 can be implemented as a server at data center 144.

[0067] In one example, device 700 may include processor 702 and / or memory 704 configured to execute or store instructions or other parameters related to providing operating system 706, which may execute one or more applications or processes, such as, but not limited to, a routing controller 250 for configuring routing between partner network 210 and data center 144 of hosting service 146. For example, processor 702 and memory 704 may be discrete components coupled by bus communication (e.g., on a motherboard or other part of a computing device, on an integrated circuit, such as a system-on-a-chip (SoC)), components integrated internally to each other (e.g., processor 702 may include memory 704 as an onboard component), and / or similar components. Memory 704 may store instructions, parameters, data structures, etc., for use / execution by processor 702 to perform the functions described herein.

[0068] In one example, the routing controller 250 includes an announcement component 350, a path selection component 352, an external rules component 354, and an internal routing component 356.

[0069] Figure 8 This is a flowchart illustrating an example of a method 800 for managing connectivity between a partner network and a WAN. For example, method 800 may be performed by device 700 and / or one or more of its components to establish a path between the partner network and the WAN, and to select a business path for services hosted on the WAN for the partner network.

[0070] In block 810, method 800 includes advertising multiple unicast border gateway protocol address prefixes for multiple front-end devices of the WAN. In one example, routing controller 250 and / or advertising component 350, for example, in conjunction with processor 702, memory 704, and operating system 706, may advertise multiple unicast border gateway protocol address prefixes 243 for multiple front-end devices 142 of WAN 240. In some implementations, each of the multiple unicast border gateway protocol address prefixes 243 identifies the corresponding front-end device 142 at a point of presence (POP) or peering connection between WAN 240 and ISP 122. In some implementations, in subblock 812, block 810 may optionally include selecting multiple front-end devices of the WAN based on the WAN network topology.

[0071] In block 820, method 800 includes selecting a path within the WAN for services for users of the partner network, the path including selected devices among a plurality of front-end devices. In one example, path selection component 352 and / or routing controller 250, for example, in conjunction with processor 702, memory 704, and operating system 706, can select paths 260, 262 within the WAN 240 for services 146 for users of the partner network 210. Paths 260, 262 include selected devices among a plurality of front-end devices 142 (e.g., front-end device 142a).

[0072] In some implementations, subblock 822, block 820 optionally includes determining measurements of multiple paths 260, 262 from agent 230 to service 146 within WAN 240, each of the multiple paths 260, 262 associated with one of multiple front-end devices 142 of WAN 240. For example, path selection component 352 may receive measurements 232 of the multiple paths from agent 230 (e.g., from measurement component 310). As another example, path selection component 352 may receive measurements of the multiple paths from an external service such as an application monitoring service. In some implementations, subblock 824, block 820 optionally includes receiving measurements 232 for portions of each of the multiple paths 260, 262 between partner network 210 and the corresponding front-end device 142, and adding measurements within the WAN from front-end device 142 to service 146. For example, path selection component 352 may receive measurements 232 for portions of each of the multiple paths from agent 230 or an external service. In some implementations, path selection within WAN240 is based on the load on paths 260 and 262.

[0073] In block 830, method 800 includes exporting routing rules to an agent within the partner network, the routing rules specifying unicast address prefixes for the selected devices. In one example, external rule component 354 and / or routing controller 250, for example, in conjunction with processor 702, memory 704, and operating system 706, can export routing rule 252 to an agent 230 within the partner network 210, the routing rule 252 specifying unicast address prefix 243 for the selected devices. In some implementations, agent 230 within the partner network 210 resides at software-defined wide area network (SD-WAN) device 112. In some implementations, agent 230 within the partner network 210 is an application controlled by WAN 240. For example, the application can be executed by user equipment 104 or UPF 106 under the control of WAN 240.

[0074] In block 840, method 800 includes receiving data packets for services to the partner network at the selected device. In one example, internal routing component 356 and / or routing controller 250, such as incorporating processor 702, memory 704, and operating system 706, may configure front-end device 142a to receive data packets 320 for service 146 to partner network 210 at the selected device 142a. Front-end device 142a may receive data packets 320 for service 146 to partner network 210. In some implementations, in sub-block 842, block 840 may optionally include, at the selected device, decapsulating service-specific packets from packets addressed to the unicast address prefix for the selected device based on routing rules. In some implementations, in sub-block 844, block 840 may optionally include, according to routing rules, forwarding data packets addressed to the unicast address prefix for the selected device to the service based on the origin address within the WAN.

[0075] Figure 9 It shows including, for example Figure 7 Examples of device 900 with additional optional component details are shown. In one aspect, device 900 may include processor 902, which may be similar to processor 702 for performing processing functions associated with one or more components and functions described herein. Processor 902 may include a single processor or a set of multi-core processors. Furthermore, processor 902 may be implemented as an integrated processing system and / or a distributed processing system.

[0076] Device 900 may further include memory 904, which may be similar to memory 704, such as for storing a local version of an operating system (or components thereof) and / or applications executed by processor 902, such as routing controller 250, advertising component 350, path selection component 352, external rules component 354, and internal routing component 356. Memory 904 may include computer-usable memory types such as random access memory (RAM), read-only memory (ROM), magnetic tape, magnetic disk, optical disk, volatile memory, non-volatile memory, and any combination thereof.

[0077] Furthermore, device 900 may include a communication component 906, which provides, for example, the establishment and maintenance of communication with one or more other devices, parties, entities, etc., using hardware, software, and services as described herein. Communication component 906 may carry communication between components on device 900, as well as communication between device 900 and external devices, such as devices located across a communication network and / or devices serially or locally connected to device 900. For example, communication component 906 may include one or more buses and may further include transmit chain components and receive chain components, respectively associated with a wireless or wired transmitter and receiver operable for interconnecting with external devices.

[0078] Furthermore, device 900 may include data storage 908 providing massive storage for information, databases, and programs used in conjunction with the aspects described herein, which may be any suitable combination of hardware and / or software. For example, data storage 908 may be, or may include, a data repository for operating systems (or components thereof) not currently executed by processor 902, applications, related parameters, etc. Additionally, data storage 908 may be a data repository for routing controller 250.

[0079] Device 900 may optionally include a user interface component 910, operable to receive input from a user of device 900, and further operable to generate output for presentation to the user. User interface component 910 may include one or more input devices, including but not limited to a keyboard, numeric keypad, mouse, touch-sensitive display, navigation keys, function keys, microphone, voice recognition component, gesture recognition component, depth sensor, gaze tracking sensor, switch / button, any other mechanism capable of receiving input from the user, or any combination thereof. Furthermore, user interface component 910 may include one or more output devices, including but not limited to a display, speaker, haptic feedback mechanism, printer, any other mechanism capable of presenting output to the user, or any combination thereof.

[0080] The device 900 may be supplemented with a routing controller 250 for managing connectivity between the partner network and the WAN, an announcement component 350 for announcing multiple unicast border gateway protocol address prefixes for multiple front-end devices in the WAN, a path selection component 352 for selecting paths within the WAN for services for users of the partner network, an external rules component for exporting routing rules to agents within the partner network, and an internal routing component 356 for receiving data packets for services of the partner network at the selected devices, etc.

[0081] For example, an element, any part of an element, or any combination of elements can be implemented using a "processing system" comprising one or more processors. Examples of processors include microprocessors, microcontrollers, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functions described throughout this disclosure. One or more processors in the processing system can execute software. Software should be interpreted broadly as instructions, instruction sets, code, code segments, program code, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description languages, or others.

[0082] Accordingly, in one or more aspects, one or more of the described functions can be implemented using hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored on a computer-readable medium or encoded as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media. Storage media can be any available medium that can be accessed by a computer. By way of example and not limitation, such computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Disks and optical discs as used herein include optical discs (CDs), laser discs, optical disks, digital versatile discs (DVDs), and floppy disks, wherein disks typically reproduce data magnetically, while optical discs reproduce data laser-like. The above combinations should also be included within the scope of computer-readable media.

[0083] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects. Therefore, the claims are not intended to limit themselves to the aspects shown herein, but should be accorded the full scope consistent with the language of the claims, wherein references to singular elements are not intended to mean “one and only one,” but rather “one or more” unless otherwise specified. Unless otherwise specified, the term “some” refers to one or more. All structural and functional equivalents of the elements of the various aspects described herein that are known to or will be known hereafter by a person skilled in the art are intended to be included by the claims. Furthermore, nothing disclosed herein is intended to be exclusive to the public, whether or not such disclosure is expressly recited in the claims. Unless an element is expressly recited using the phrase “means for…”, no claim element should be construed as means plus function.

Claims

1. An apparatus for communication between a partner network and services hosted in a wide area network (WAN) connected to the partner network via the Internet, wherein the apparatus is a software-defined wide area network (SD-WAN) device within the partner network, the partner network being a computer network connected to the WAN via the Internet, the apparatus comprising: A memory that stores one or more instructions for managing routing for the partner network; as well as At least one processor, coupled to the memory and configured to execute the instructions, wherein the at least one processor is configured to: Measure multiple paths from the partner network to the service within the WAN to generate a measurement for each of the multiple paths, wherein each of the multiple paths is associated with one of a plurality of front-end devices of the WAN, the plurality of front-end devices being associated with a corresponding unicast border gateway protocol address prefix, and wherein the measurement includes at least one of the following: latency, delay, throughput, devices traversed, and a trajectory route indicating one or more networks between the partner network and the WAN; Provide the WAN with measurements of the multiple paths to the service; The routing rules are received from the WAN, which specify unicast address prefixes for selected devices among the plurality of front-end devices of the WAN, wherein the selected front-end devices are associated with paths with better characteristics identified by the WAN from the plurality of paths based on measurements provided by the SD-WAN devices; as well as Data packets for the service are forwarded via the Internet to the corresponding Border Gateway Protocol address prefix of the selected front-end device.

2. The apparatus of claim 1, wherein each of the respective unicast border gateway protocol address prefixes identifies a point of presence (POP) or peer-to-peer connection between the WAN and the Internet Service Provider (ISP).

3. The apparatus of claim 1, wherein the partner network is an enterprise network or includes a radio access network.

4. The apparatus of claim 1, wherein the one or more instructions for managing routing to the partner network define an application controlled by the WAN.

5. The apparatus of claim 1, wherein the at least one processor is configured to encapsulate packets for the service within packets for the unicast address prefix of the selected front-end device based on the routing rules.

6. The apparatus of claim 1, wherein the at least one processor is configured to rewrite the header of packets for the service based on the routing rules and a port corresponding to the original address for the service, using the unicast address prefix for the selected front-end device.

7. The apparatus of claim 1, wherein measuring the plurality of paths of the service within the WAN comprises: Ping packets are sent to the service along each of the plurality of available paths to determine the latency associated with each path.

8. The apparatus of claim 1, wherein the at least one processor is further configured to: receive a list of unicast border gateway protocol address prefixes from the routing controller of the WAN.

9. The apparatus of claim 1, wherein the at least one processor is configured to measure a portion of each path to a corresponding one of the plurality of front-end devices of the WAN via the respective unicast border gateway protocol address prefix.

10. A method for communication between a partner network and services, the services being hosted in a wide area network (WAN) connected to the partner network via the Internet, wherein the method is performed by a software-defined wide area network (SD-WAN) device within the partner network, the partner network being a computer network connected to the WAN via the Internet, the method comprising: Measure multiple paths from the partner network to the service within the WAN to generate a measurement for each of the multiple paths, wherein each of the multiple paths is associated with one of a plurality of front-end devices of the WAN, the plurality of front-end devices being associated with a corresponding unicast border gateway protocol address prefix, and wherein the measurement includes at least one of the following: latency, delay, throughput, devices traversed, and a trajectory route indicating one or more networks between the partner network and the WAN; Provide the WAN with measurements of the multiple paths to the service; Receive routing rules from the WAN, the routing rules specifying unicast address prefixes for selected front-end devices among the plurality of front-end devices of the WAN, wherein the selected front-end devices are associated with paths with better characteristics identified by the WAN from the plurality of paths based on measurements provided by the SD-WAN device; and Data packets for the service are forwarded via the Internet to the corresponding Border Gateway Protocol address prefix of the selected front-end device.

11. The method of claim 10, wherein each unicast border gateway protocol address prefix in the respective unicast border gateway protocol address prefix identifies a point of presence (POP) or peer-to-peer connection between the WAN and the Internet Service Provider (ISP).

12. The method of claim 10, wherein the partner network is an enterprise network or includes a radio access network.

13. The method of claim 10, wherein receiving the routing rule occurs at the application controlled by the WAN.

14. The method of claim 10, wherein forwarding the data packet for the service comprises: Based on the routing rules, packets for the service are encapsulated within packets containing the unicast address prefix for the selected front-end device.

15. The method of claim 10, wherein forwarding the data packet for the service comprises: Based on the routing rules and the port corresponding to the original address for the service, the header of the packet for the service is rewritten using the unicast address prefix for the selected front-end device.

16. The method of claim 10, wherein the plurality of paths to the service within the WAN include latency: sending ping packets to the service along each available path of the plurality of paths to determine the latency associated with each path.

17. The method of claim 10, further comprising: Receive the list of unicast border gateway protocol address prefixes from the routing controller of the WAN.

18. The method of claim 10, wherein the plurality of paths measured to the service within the WAN comprise a portion of each path measured to a corresponding one of the plurality of front-end devices in the WAN via the respective unicast border gateway protocol address prefix.

19. A non-transitory computer-readable medium comprising stored instructions for communication between a partner network and services hosted in a wide area network (WAN) connected to the partner network via the Internet, wherein the partner network is a computer network connected to the WAN via the Internet, the instructions being executable by a processor of a software-defined wide area network (SD-WAN) device within the partner network to: Measure multiple paths from the partner network to the service within the WAN to generate measurements for each of the multiple paths, wherein, Each of the plurality of paths is associated with one of a plurality of front-end devices of the WAN, the plurality of front-end devices being associated with a corresponding unicast border gateway protocol address prefix, and wherein the measurement includes at least one of the following: latency, delay, throughput, devices traversed, and a trajectory route indicating one or more networks between the partner network and the WAN; Provide the WAN with measurements of the multiple paths to the service; The routing rules are received from the WAN, which specify unicast address prefixes for selected devices among the plurality of front-end devices of the WAN, wherein the selected front-end devices are associated with paths with better characteristics identified by the WAN from the plurality of paths based on measurements provided by the SD-WAN devices; as well as Data packets for the service are forwarded via the Internet to the corresponding Border Gateway Protocol address prefix of the selected front-end device.

20. The non-transitory computer-readable medium of claim 19, wherein each of the respective unicast border gateway protocol address prefixes identifies a point of presence (POP) or peer-to-peer connection between the WAN and the Internet Service Provider (ISP).

Images