Opaque routing on Overlay network: method based on structured neural network

Abstract

In some embodiments, a method may be provided that includes receiving at least a first traffic matrix as a first input to a first machine learning model, the first traffic matrix indicating an amount of traffic routed between at least one pair of nodes of an Overlay network; receiving, as a second input to the first machine learning model, information indicating an Overlay network route between at least one node pair of the Overlay network; receiving a measurement delay between at least one node pair of the Overlay network as a third input to the first machine learning model; and learning a representation of the Underlay network by a first machine learning model, the learning using a minimization of a difference between an average delay in the Underlay network and a measured delay between at least one pair of nodes of the Overlay network.

Description

Technical Field

[0001] The topics covered in this article relate to networking and machine learning. Background Art

[0002] Machine learning (ML) models can learn via training. ML models can take many forms, such as artificial neural networks (or simply neural networks), decision trees, etc. The training of ML models can be supervised (using labeled training data), semi-supervised, or unsupervised. When trained, ML models can be used to perform reasoning tasks. Summary of the invention

[0003] In some embodiments, a method may be provided, comprising: receiving at least a first traffic matrix as a first input to a first machine learning model, the first traffic matrix indicating the traffic routed between at least one node pair of an overlay network; receiving information indicating the following as a second input to the first machine learning model: Overlay network routing between at least one node pair of the overlay network; receiving a measured delay between at least one node pair of the overlay network as a third input to the first machine learning model; learning a representation of an underlay network by the first machine learning model, the learning using minimization of the difference between an average delay in the underlay network and a measured delay between at least one node pair of the overlay network; and outputting a representation of the underlay network by the first machine learning model.

[0004] In some variations, one or more features disclosed herein (including the following features) may optionally be included in any feasible combination. Learning may also include learning a representation of background traffic in an Underlay network by a first machine learning model, the learning of the representation of background traffic in the Underlay network using the minimization of the difference between the average delay in the Underlay network and the measured delay between at least one node pair of the Overlay network. The output may also include: outputting a representation of background traffic in the Underlay network. The second machine model may learn one or more routing parameters for the Overlay network, the learning using at least a representation of the Underlay network and / or a representation of background traffic in the Underlay network. The second machine learning model may receive at least a second traffic matrix for the Overlay network as a first input to the second machine learning model. The second machine learning model may receive a representation of the Underlay network as a second input to the second machine learning model. The second machine learning model may receive a representation of background traffic in the Underlay network as a third input to the second machine learning model. The second machine learning model may learn a representation of one or more routing parameters for the Overlay network by minimizing the average delay of the node pairs of the Overlay network. The second machine learning model may output a representation of one or more routes for the Overlay network to implement routing configuration in the Overlay network. The second machine learning model may include: an input layer that receives the following items: the second service matrix for the Overlay network, the representation of the Underlay network, and the representation of the background service in the Underlay network, an intermediate layer that determines the following items: one or more intermediate values ​​for one or more segment flows, one or more link flows, one or more link delays, and one or more calculated delays, and an output layer that outputs the following items to implement routing configuration in the Overlay network: the representation of the one or more routing parameters for the Overlay network. The first machine learning model may include: an input layer that receives the following items: the first input, the second input, and the third input, an intermediate layer that determines the following items: one or more intermediate values ​​for one or more segment flows, one or more link delays, and one or more calculated delays, and an output layer that outputs the following items: the representation of the Underlay network.

[0005] The above aspects and features can be implemented in systems, devices, methods and / or articles depending on the desired configuration. The details of one or more variations of the subject matter described herein are set forth in the accompanying drawings and the following description. The features and advantages of the subject matter described herein will be apparent from the description and drawings as well as the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] In the accompanying drawings,

[0007] Figure 1A An example of an Overlay network located on an Underlay network according to some embodiments is shown;

[0008] Figure 1B It shows how the services in the Underlay network are mapped to the services in the Overlay network according to some embodiments;

[0009] Figure 1C Depicting an example of segment routing between two nodes i and j of an overlay network according to some embodiments;

[0010] Figure 1D depicts an example of 2-hop segmentation according to some embodiments;

[0011] Figure 1E depicts examples of segmented segment routing and non-segmented segment routing according to some embodiments;

[0012] Figure 2A An example of training a machine learning (ML) model according to some embodiments is depicted, the model being configured to learn a representation of an underlay network based at least on measured delays on an overlay network;

[0013] Figure 2B Depicting an example of training a machine learning (ML) model used to optimize routing in an overlay network when an opaque underlay network is present according to some embodiments;

[0014] Figure 3A Depicting an example of a process for training an ML model so that the ML model learns a representation of an opaque Underlay network according to some embodiments;

[0015] Figure 3B Depicting an example of a process for optimizing routing of an overlay network using an ML model according to some embodiments;

[0016] Figure 4 depicts an example of an ML model according to some embodiments;

[0017] Figure 5 depicts an example of a network node according to some embodiments; and

[0018] Figure 6 An example of a computing system in accordance with some embodiments is depicted.

[0019] The same reference numbers are used in the drawings to refer to the same or similar items. DETAILED DESCRIPTION

[0020] Figure 1A An Underlay network 150A and an Overlay network 154A located on the Underlay network 150A are shown. The Underlay network may include a plurality of nodes 150B-F coupled by one or more links 151A-F, and data (such as packets) may be routed between the nodes of the Underlay network using, for example, shortest path routing. Also, the Overlay network may include a plurality of nodes 154B-F coupled by one or more segments 155A-G (each segment may include one or more Underlay links), and segment routing may be used between the nodes. For example, the Underlay network may correspond to a physical network coupled by physical links that route data using shortest path routing, and the Overlay network may correspond to a virtual private network located on the physical Underlay network.

[0021] exist Figure 1A In the example of , a user of Overlay network 154A may be able to control routing using, for example, segmented routing between nodes 154B-F, but the user of the Overlay network may know little or nothing about Underlay network 150A. In other words, from the perspective of the Overlay network, the Underlay network is "opaque" to Underlay network details (such as topology, routing being used, actual performance of Underlay nodes, actual performance of Underlay links, and / or other network information about the physical Underlay network). For example, a service provider will typically not allow users of an Overlay network (such as a VPN) to access information about the physical network of the network service provider that provides Underlay network 150A. Service providers will also typically not allow users of the Overlay network to control routing or configuration of the Underlay network. To further illustrate, assume that a user of the Overlay network attempts to reroute traffic to improve performance or avoid nodes of the Overlay network by rerouting traffic between node 154B and node 154F. Although the Overlay network user may not be able to control routing in the Underlay network, the user can configure routing via segments in the Overlay routing. However, since the underlay network is opaque, the user is unaware of any impact(s) that such changes in the overlay network may have on the underlay network. For example, such rerouting may (or may not) provide the performance improvement expected or desired by the user.

[0022] Figure 1B Depicted Figure 1A The Underlay network 150A and Overlay network 154A in FIG. 1 further illustrate how data traffic in the Underlay network is mapped to the Overlay network. Figure 1B In the example of FIG. 154B , traffic in the overlay network between node i 154B and node j 154E is routed through segment 155E. In this case, traffic can be routed through the underlay network via one or more physical links (such as links 151D and 151E and links 151C and 151F). Traffic in the underlay network is routed by β ij (l) represents, where β ij (l) represents the fractional amount of traffic on physical link l when the unit flow is sent between nodes i and j in the overlay network (e.g., via segment 155E). Figure 1B In the example of FIG. 1 , traffic leaving node 150B is split so that half of the traffic traverses links 151D and 151E, and the other half traverses links 151C and 151F. In this example, links 151D and 151E each have β ij (l) = 1 / 2, and links 151C and F each have β ij (l) = 1 / 2. In other words, β ij (l) represents the fraction of traffic from source node i to destination node j that is routed on a given link l.

[0023] In the overlay network 154A, the amount of traffic (also called traffic demand) routed between node i 154B and node j 154E in the overlay network at time t is given by d ij (t). This overlay traffic can be routed from node i to node j along multiple hops on the overlay network using segment routing (such as single deflection segment routing (SDSR)). As previously described, this overlay routing will cause traffic in the links of the underlay network, and the corresponding delay of each link in the underlay network is at least a function of the utilization of the link (although users of the overlay network will not have actual latency to the physical links of the underlay network because it is opaque). Of the traffic sent from node i to node j, the fraction of traffic that is deflected by an intermediate node k on the overlay network (e.g., node 154F) can be represented by the following express:

[0024]

[0025] In the Underlay network 150A, the background traffic on a given link l in the Underlay network can be represented by λ(l). Moreover, the background traffic λ(l) can be considered independent of any traffic generated by the Overlay network. For example, the background traffic λ(l) can be other traffic carried by the Underlay network that is not associated with the Overlay network 154A.

[0026] Since the Underlay network is not transparent to the (multiple) users of the Overlay network, a user of the Overlay network 154A may not have information about the Underlay network 150A, but the Overlay network user may still measure the delay between any two nodes of the Overlay network. For example, the delay between node i 154B and node j 154E (as well as other pairs of nodes in the Overlay network) can be measured by sending a packet with a timestamp on the link or segment 155E (e.g., using a ping or other type of probe). The delay can be a unit of time, such as milliseconds, and the delay can represent an average delay, in which case multiple delay measurements are performed between the nodes and then averaged.

[0027] In some embodiments, a method for routing on an overlay network (such as a virtual private network) is provided without having information about the topology and / or routing of an "opaque" underlay network. For example, traffic (without knowledge of the underlay topology and / or routing) can be routed on the overlay network, and one or more delays between pairs of nodes of the overlay network can be measured. Next, a machine learning model can be used to learn how to model (e.g., represent) the topology and / or routing of the underlay network, and such learning can be based at least on one or more measured delays between pairs of nodes of the overlay network. When the machine learning (ML) model is trained, the machine learning model can provide a representation of the topology and / or routing of the underlay network as an output. The ML model output may include routing parameters β of the underlay network. ij (l). This output can then be used by an ML model to determine how to best route traffic on the overlay network.

[0028] Before providing additional details about learning underlay network topology and routing based on ML models and optimizing routing on overlay networks, additional details about segment routing on overlay networks, latency, underlay network routing (e.g., βij(l)), and the objective function of the ML model using latency are provided below.

[0029] Figure 1C Depicts an example of using segmentation to perform segment routing between two nodes i and j in an overlay network. Figure 1C In the example of , there is a first segment 102A between node i 154B and node k 154F, and a second segment 102B between node k 154F and node j 154E (where node k can be referred to as an intermediate node or a deflection node). Within a given segment (such as segment 102A or 102B of an overlay network), the shortest path routing can be used to determine one or more physical links l (for example, in an underlay network), and one or more physical links l are used to physically carry traffic (such as packet 106) in the underlay network. Formally, a segment can represent a minimum weight path between a pair of nodes. For example, a segment between nodes i and j can be a minimum weight path between nodes i and j, where the minimum weight path is calculated using a link metric.

[0030] refer to Figure 1C 108A and destination address "j" 108B. The segment label specifies to node i 154B that packet 106 is to be forwarded to destination node j 154E through segment 102A-B, for example using segment-based routing but deflected via intermediate node k 154F, while the physical links (within the segment) and associated link metrics may be selected using a shortest path algorithm through the links of segment 102A-B. For example, within each segment, shortest path routing may be used to determine a path through the physical links of the underlay network to carry packet 106 to intermediate node k 154F.

[0031] To further illustrate by numerical example, the traffic through any given segment between nodes i and j ("segment traffic flow") may be represented as follows:

[0032]

[0033] The segmented traffic flow is φ ij , and is the total traffic flow on the segment between node i and node j, and the deflection parameter is and And d kj (t) and dik (t) indicates the traffic demand (e.g., obtained from the traffic matrix). In other words, the traffic deflected via node k (between node i and node j) is a function of the deflection parameter and the number of traffic flows on the segment. For example, for The deflection parameter is 0.75, and given that the traffic demand between nodes i and j is 10 GB, the traffic flow on the segment passing through node k will be 7.5 GB. Figure 1D As shown, the amount of traffic flow on the segment is calculated for the segment between nodes i and j, and the traffic flow takes into account that the segment may be a two-hop second hop segment 168A and a two-hop first hop segment 168B.

[0034] Due to the unit flow on the segment between node i and node j in the Overlay network, the flow on link l in the Underlay network can be expressed as β ij (l), and the background traffic on link l can be represented by λ(l). The background traffic in the Underlay service is independent of the traffic in the Overlay network segment. The total traffic F(l,t) on a given link l at time t can be represented by:

[0035]

[0036] As mentioned above, the delay in a link ("link delay") is an increasing function of the link utilization. Therefore, the delay δ(l,t) on link l at time t can be expressed as follows:

[0037]

[0038] Where c(l) is the capacity of link l.

[0039] Regarding the average delay between nodes in the Overlay network, the average delay between node i and node j in the Overlay network is a function of the average delay on the path between node i and node j on the Underlay network. The average delay on the path between node i and node j at time t in the Underlay network can be expressed as and is represented by:

[0040]

[0041] Average latency Can be measured between some (if not all) node pairs in the overlay network.

[0042] Underlay network routing can be indicated by βij(l) values, as these values ​​encapsulate information about the underlay topology and routing. To train the ML model, the learning can use an objective (or loss) function to determine the βij(l) value, which, for example, minimizes the average delay using the following The difference between the latency measured in the Overlay network and the latency measured in the Underlay network (which is unknown to the Overlay network because the Underlay network is opaque) is ω ij (t):

[0043]

[0044] For example, a machine learning model (e.g., using the PyTorch framework) can use the objective function (using stochastic gradient descent, as described in Figure 2A ) is learned to determine a model (e.g., representation) of the underlay network without having information about the underlay network.

[0045] Figure 2A An example of a machine learning model 160 is depicted for learning a representation of an underlay network without knowledge of the topology and / or routing of the underlay network. The ML model (a neural network in this example) is trained using a traffic matrix and corresponding measured delays on pairs of nodes of the overlay network. In an example training session of the ML model, a total of approximately 100 training samples are used, and 10 batches are randomly selected from the 100 training samples for training (although other training set sizes and batch sizes may also be used).

[0046] In ML model 160, the decision variable in ML model learning is the Underlay network routing β ij (l) parameter 210 and background traffic λ(l) parameter 212. At each step during ML model training, a subset of the traffic matrix and its corresponding overlay delays are fed into the ML model. The ML model adjusts the decision variables β based on an objective function (which attempts to converge to the minimum difference between the calculated delay and the observed delay described above with respect to Equation 6 and shown at 215) ij (l) and λ(l).

[0047] According to some embodiments, at 202A-C, the ML model 160 may receive one or more traffic matrices as input. For example, the traffic matrix 202A may indicate the traffic volume routed between any two nodes (such as nodes i and j) in the overlay network 154A at time t. As previously described, the traffic volume between nodes i and j in the overlay network 154A may be represented by d ij(t) indicates that the traffic matrix 202A may include traffic between pairs of nodes in the Overlay network 154. Figure 1A In the example, there are 5 nodes and the service matrix includes ij (t) 25 values ​​of node pairs.

[0048] At 204, according to some embodiments, the ML model 160 may receive information about overlay routing as input. For example, the ML model may receive values ​​for segment routing parameters, such as deflection parameters It indicates the topology and routing used on the Overlay network.

[0049] At 206, according to some embodiments, the ML model 160 may receive as input information about measured delays between nodes of the overlay network. For example, the ML model may measure the delay between a pair of nodes i and j of the overlay network. For example, referring to Figure 1A For the Overlay network 154A, the ML model may receive 25 measured delay values ​​(ω) for 5 nodes of the Overlay network. ij (t)), for the corresponding service matrix d ij (t).

[0050] Given the inputs at 202A-C, 204, and 206, the ML model 160 may learn the routing 210 of the underlay network (e.g., the underlay network routing β ij (l) parameter) and / or background traffic 212 (e.g., background traffic λ(l) parameter), which minimizes the average delay at 215 value (not called the Underlay network and its links are not transparent to the Overlay network) and the measured delay ω ij (t) values ​​(measured by the Overlay network), for example using Equation 6 above. For example, a decision variable (such as β ij The λ(l) parameter 210 and the background traffic λ(l) parameter 212) may be varied using back-propagation until the objective function converges to a minimum difference between the calculated delay and the observed delay at 215.

[0051] like Figure 2A As shown, the ML model 160 computes as part of the learning the φ for the segmented stream 220 (e.g., see φ at Eq. 2). ij ), link flow 222 (e.g., see F(l,t) at Equation 3), link delay 224 (e.g., see δ(l,t) at Equation 4), and calculated delay 226 (e.g., see These intermediate values ​​are used to determine the difference at 215.

[0052] exist Figure 2A In the example of , the machine learning model 160 has a structure including an input layer (e.g., of one or more computing nodes) configured to receive a first input, a second input, and a third input (e.g., inputs 202A-C, 204, and 206). The machine learning model may also include an intermediate layer (e.g., of one or more computing nodes) configured to determine one or more intermediate values ​​for one or more segment flows, one or more link delays, and one or more computing delays (e.g., 220, 222, 224, and 226). The machine learning model may also include an output layer (e.g., of one or more computing nodes) configured to output a representation of the Underlay network (e.g., 210).

[0053] When the ML model converges and minimizes the average latency of the computation at 215 The measured delay ω on the traffic matrix 202A-C ij (t) and the measured delay 206 value, Underlay routing β ij (l) 210 Parameters (e.g., values) are used as a model or representation of the underlay network. In other words, the underlay routing β ij The (l) value provides a representation of the Underlay topology and its routing. As mentioned above, β ij (l) value can model the topology and routing of the Underlay network, and β ij The (l) value may (or may not) indicate the actual routing value used in the opaque underlay network. In addition, since the underlay network is opaque, the number of nodes in the underlay may be unknown, so the ML model can use the initial number and continue to learn β ij (l).

[0054] Once the Underlay topology uses the Underlay routing parameter β ij (l) 210 Learning, the Underlay information can be used to optimize the service routing on the Overlay network (for example, to minimize the average delay of the routing). For example, assuming that d ij represents the service matrix that must be routed through the Overlay network, β^ ij (l) and λ^(l) represent the β learned by the ML model 160 ij (l)210 and λ(l)212 values, the decision variables are It represents the overlay routing parameters, such as the portion of traffic from node i to j that is diverted through node k, constrained as

[0055]

[0056] This constraint can be reformulated as an unconstrained optimization for ML models. For example, can be defined so that where α>0 is a fixed constant, so the constraint of Equation 7 means

[0057]

[0058] Any of these This transformation ensures and is satisfied. Through this reformulation, the traffic on link l in the Underlay network is expressed as follows:

[0059]

[0060] The delay on link l in the underlay network can be expressed as follows:

[0061]

[0062] And, the average delay between node i and node j in the Overlay network can be expressed as follows:

[0063]

[0064] The overall optimization problem of minimizing the average latency can be expressed as follows:

[0065]

[0066] The variables β^ij(l) and λ^(l) are estimated from ML learning using, for example, the PyTorch machine learning framework.

[0067] Figure 2B Another example of an ML model 260 is depicted, which is used to optimize traffic in an overlay network when the underlay network is opaque, such that the overlay network (or its users) at least has no knowledge about the topology and / or routing of the underlay network, according to some embodiments.

[0068] At 252, according to some embodiments, the ML model 260 may receive at least one traffic matrix as input. For example, the traffic matrix 252 may indicate the amount of traffic routed between any two nodes (such as nodes i and j) in the overlay network 154A at time t.

[0069] At 254A, according to some embodiments, the ML model 260 may receive information about underlay routing as input. For example, the ML model may receive Figure 2A Underlay network routing learned in β ij (l) parameters; the learned Underlay network routing parameters are denoted as β^ij(l).

[0070] According to some embodiments, at 254B, the ML model 260 may receive as input information about background traffic in the underlay network. For example, the ML model may receive Figure 2A The background service λ(l) parameters learned in , and are expressed as the learned background service λ^(l).

[0071] Given the inputs at 252, 254A, and 254B, the ML model 260 can learn the routing 270 parameters of the overlay network (e.g., value), which minimizes the objective function at 280, which corresponds to the average delay. At 280, the calculated delay Δ ij Based on business ij For example, the routing parameters 270 of the Overlay network (e.g., value) can be changed using back-propagation until the objective function converges to the minimum difference between the calculated delay and the observed delay at 280.

[0072] like Figure 2B As shown, the ML model 260, as part of the learning, calculates the φ for the segmented stream 220 (e.g., see φ at Eq. 2). ij ), link flow 222 (e.g., see F(l,t) at Equation 3), link delay 224 (e.g., see δ(l,t) at Equation 4), and calculated delay 226 (e.g., see ). When the ML model 260 converges by finding the minimum value for the average delay at 280, the optimal overlay route Parameter 270 can be used as an output to configure segment routing on the overlay network.

[0073] exist Figure 2BIn the example, the machine learning model 260 has a structure including an input layer (e.g., of one or more computing nodes), which is configured to receive a second service matrix for the Overlay network, a representation of the Underlay network, and a representation of the background service in the Underlay network (e.g., 252 and 254A-B). The machine learning model may also include an intermediate layer (e.g., of one or more computing nodes), which is configured to determine one or more intermediate values ​​for one or more segment flows, one or more link flows, one or more link delays, and one or more calculation delays (e.g., 220, 222, 224, and 226). Furthermore, the machine learning model may also include an output layer (e.g., of one or more computing nodes), which is configured to output a representation of one or more routing parameters (e.g., 270) of the Overlay network to implement configuration of routing in the Overlay network.

[0074] In some implementations, the unsegmented SDSR routing scheme may be derived from the segmented routing Overlay parameters provided at 270. In the case of segmented routing, traffic from source node i to destination node j is segmented between multiple deflection points, such as Figure 1E 175A. At 175A, part of the traffic from source node i to destination node j is deflected through k1, and the rest of the traffic is deflected through node k2. In the case of unsegmented SDSR, the traffic from source node i to destination node j is deflected through a single deflection node k, as shown in 175B. As shown in 175B, in the case of unsegmented SDSR, all traffic from source node i to destination node j is deflected through a single node k.

[0075] In order to derive the unsegmented SDSR from the segmented SDSR, the corresponding It is determined as follows:

[0076]

[0077] And, randomized rounding is performed. For example, for each ij pair, a k is picked by, for example, rolling a so-called n-sided die, where the probability of obtaining k is because So there is a valid probability distribution. Next, if and only if k* is obtained when ij is rolled, After rounding all ij, we get represents the undivided SDSR. This rounding process can be repeated multiple times, and the solution that results in the lowest maximum link utilization can be picked.

[0078] Figure 3ADepicted is an example of a process for training an ML model in accordance with some embodiments, such that the ML model learns a representation of an opaque underlay network.

[0079] At 305, according to some embodiments, the ML model may receive as a first input one or more traffic matrices. For example, the traffic matrix 202A may be received, and the traffic matrix may indicate the amount of traffic routed between any two nodes in an overlay network (such as overlay network 154A) at a given time t.

[0080] According to some embodiments, at 307, the ML model may receive information about the overlay network routing as a second input. For example, the second input may include segment routing parameters for indicating the topology and routing used on the overlay network. The value of .

[0081] According to some embodiments, at 309, the ML model may receive information about measured delays between nodes of the overlay network as a third input. For example, the ML model 160 may receive 25 measured delay values ​​(ω ) for 5 nodes of the overlay network 154A. ij (t)), for the corresponding service matrix d ij (t).

[0082] According to some embodiments, at 311, the ML model may learn a representation of the Underlay network. For example, given inputs 305-309, the ML model 160 may learn routing parameters β of the Underlay network. ij (l), which minimizes the average delay The value (unknown because the Underlay network is opaque) is related to the measured delay ω ij Additionally or alternatively, given inputs 305-309, ML model 160 may learn background traffic λ(l) parameters for the Underlay network that minimize the average delay Value and measurement delay ω ij (t) The difference between the values.

[0083] When the ML model converges and minimizes the average latency of computation Measuring Delay ω in Underlay ij (t), the machine learning model 160 may output a representation of the Underlay network at 313, such as the Underlay routing β ijAdditionally or alternatively, the ML model 160 may output background traffic λ(l) parameters 212. A representation of the underlay network (e.g., underlay routing β ij The background traffic λ(l) parameter 210) and / or the background traffic (eg, the background traffic λ(l) parameter 212) may be output for use in optimizing routing of the Overlay network.

[0084] Figure 3B Depicted is an example of a process for optimizing routing of an overlay network using an ML model in accordance with some embodiments.

[0085] At 345, according to some embodiments, an ML model (such as ML model 260) may receive as a first input one or more traffic matrices. For example, the first input may include at least one traffic matrix indicating the amount of traffic routed between any two nodes in an overlay network (such as overlay network 154A) at a given time t.

[0086] At 350, according to some embodiments, the ML model (such as ML model 260) may receive as a second input information about the Underlay route learned by the ML model (such as ML model 160). For example, the first input may include the learned Underlay network route β^ij(l) provided at 210, as Figure 2A The output of the ML model 160.

[0087] According to some embodiments, at 352, the ML model (such as ML model 260) may receive information about the learned background traffic in the underlay network as a third input. For example, the second input may include the learned background traffic λ^(l) provided at 212, as Figure 2A The output of the ML model 160.

[0088] Given the inputs at 345-352, the ML model 260 may learn at 355 the routing 270 parameters of the overlay network that minimize the objective function (e.g., value 270 ). For example, ML model 260 may minimize average latency, as described at 280 .

[0089] When the ML model 260 converges and minimizes the average delay 280, the machine learning model 260 may output the routing 270 parameters of the overlay network at 360. The routing 270 parameters of the overlay network may be used to configure segment routing on the overlay network.

[0090] Figure 4An example of an ML model 400 according to some embodiments is shown. The ML model 400 may be used as the ML model 160. The input layer 410 may include a node for each node in the network. The ML model may include one or more hidden layers 415A-B (also referred to as intermediate layers) and an output layer 420. The machine learning model 400 may be included in a network node, a user device, and / or other computer-based system. Additionally or alternatively, the ML model may be provided as a service, such as a cloud service (accessible at a computing system such as a server via a network such as the Internet or other types of networks).

[0091] Figure 5 A block diagram of a network node 500 according to some embodiments is shown. As previously described, the machine learning model 160 or 260 may be included in the network node. The network node 500 may be included or contained in one or more network-side nodes or functions. According to some embodiments, the network node 500 may include a network interface 502, a processor 520, and a memory 504. The network interface 502 may include a wired and / or wireless transceiver to enable access to other nodes, including base stations, other network nodes, the Internet, other networks, and / or other nodes. The memory 504 may include volatile and / or non-volatile memory, including program code, which, when executed by at least one processor 520, provides, among other things, the processes disclosed herein. For example, a network node (such as a network management system) may include an ML model 160 and / or 260 to learn Underlay network routing parameters and optimize Overlay routing when the Underlay network is not transparent.

[0092] Figure 6 1 depicts a block diagram of a computing system 700 according to some embodiments. For example, the network management system 150 and / or the ML model 160 (or 400) may include the system 700. Figure 6As shown, the computing system 700 may include a processor 710, a memory 720, a storage device 730, and an input / output device 740. The processor 710, the memory 720, the storage device 730, and the input / output device 740 may be interconnected via a system bus 750. The processor 710 is capable of processing instructions for execution within the computing system 700. In some implementations of the current subject matter, the processor 710 may be a single-threaded processor. Alternatively, the processor 710 may be a multi-threaded processor. The process may be a multi-core processor or a single-core processor having multiple processors. Additionally or alternatively, the processor 710 may be a graphics processor unit (GPU), an AI chip, etc. The processor 710 is capable of processing instructions stored in the memory 720 and / or the storage device 730 to display graphical information for a user interface provided via the input / output device 740. The memory 720 is a computer-readable medium, such as volatile or non-volatile, which stores information within the computing system 700. For example, the memory 720 may store a data structure representing a configuration object database. Storage device 730 can provide persistent storage for computing system 700. Storage device 730 can be a floppy disk device, a hard disk device, an optical disk device or a magnetic tape device, or other suitable persistent storage components. Input / output device 740 provides input / output operation for computing system 700. In some implementations of current theme, input / output device 740 includes a keyboard and / or a pointing device. In various implementations, input / output device 740 includes a display unit for displaying a graphical user interface. According to some implementations of current theme, input / output device 740 can provide input / output operation for network equipment. For example, input / output device 740 can include an Ethernet port or other network ports to communicate with one or more wired and / or wireless networks (e.g., local area network (LAN), wide area network (WAN), the Internet).

[0093] Without in any way limiting the scope, interpretation, or application of the claims below, the technical effects of one or more of the example embodiments disclosed herein may include enhanced network optimization so that the network can more efficiently route traffic on the overlay network without knowledge of the topology and / or routing of the underlay network.

[0094] In view of the implementation of the above-mentioned subject matter, the present application discloses the following list of examples, wherein one feature of the example alone or one or more features of the example combined and optionally combined with one or more features of one or more further examples are further examples disclosed that also belong to the present application.

[0095] Example 1: A method, comprising: receiving at least a first traffic matrix as a first input to a first machine learning model, the first traffic matrix indicating the amount of traffic routed between at least one node pair of an overlay network; receiving information indicating the following as a second input to the first machine learning model: Overlay network routing between the at least one node pair of the overlay network; receiving a measured delay between the at least one node pair of the overlay network as a third input to the first machine learning model; learning a representation of an underlay network by the first machine learning model, the learning using minimization of the difference between an average delay in the underlay network and the measured delay between the at least one node pair of the overlay network; and outputting the representation of the underlay network by the first machine learning model.

[0096] Example 2. A method according to Example 1, wherein the learning further includes: learning a representation of background traffic in the Underlay network by the first machine learning model, wherein the learning of the representation of the background traffic in the Underlay network uses the minimization of the difference between the average delay in the Underlay network and the measured delay between at least one node pair of the Overlay network.

[0097] Example 3. The method according to Examples 1 to 2, wherein the output further comprises: outputting the representation of the background traffic in the underlay network.

[0098] Example 4. The method according to Examples 1 to 3 further includes: learning one or more routing parameters for the overlay network by a second machine model, wherein the learning uses at least the representation of the underlay network and / or the representation of background traffic in the underlay network.

[0099] Example 5. The method according to Examples 1 to 4 also includes: receiving at least a second service matrix for the overlay network as a first input to a second machine learning model.

[0100] Example 6. The method according to any one of Examples 1 to 5 further includes: receiving the representation of the Underlay network as a second input to the second machine learning model.

[0101] Example 7. The method according to any one of Examples 1 to 6 further includes: receiving a representation of background traffic in the underlay network as a third input to the second machine learning model.

[0102] Example 8. The method according to any one of Examples 1 to 7 further includes: using the second machine learning model to learn representations of one or more routing parameters for the overlay network by minimizing the average delay of node pairs in the overlay network.

[0103] Example 9. The method according to any one of Examples 1 to 8 further includes: the second machine learning model outputting the representation of the one or more routes for the overlay network to implement routing configuration in the overlay network.

[0104] Example 10. A method according to any one of 1 to 10, wherein the second machine learning model includes: an input layer receiving the following items: the second service matrix for the Overlay network, the representation of the Underlay network, and the representation of the background service in the Underlay network, determining an intermediate layer of the following items: one or more intermediate values ​​for one or more segment flows, one or more link flows, one or more link delays, and one or more calculated delays, and an output layer outputting the following items to implement configuring routing in the Overlay network: the representation of the one or more routing parameters for the Overlay network.

[0105] Example 11. A method according to any one of Examples 1 to 11, wherein the first machine learning model includes: an input layer that receives the following items: the first input, the second input, and the third input, an intermediate layer that determines the following items: one or more intermediate values ​​for one or more segmented flows, one or more link delays, and one or more calculated delays, and an output layer that outputs the following items: the representation of the Underlay network.

[0106] Example 12: A device comprising: at least one processor; and at least one memory, comprising instructions, which when executed by the at least one processor cause operations, the operations comprising: receiving at least a first traffic matrix as a first input to a first machine learning model, the first traffic matrix indicating the amount of traffic routed between at least one node pair of an overlay network; receiving information indicating the following as a second input to the first machine learning model: overlay network routing between the at least one node pair of the overlay network; receiving measured delays between the at least one node pair of the overlay network as a third input to the first machine learning model; learning a representation of an underlay network by the first machine learning model, the learning using minimization of the difference between an average delay in the underlay network and the measured delay between the at least one node pair of the overlay network; and outputting the representation of the underlay network by the first machine learning model.

[0107] Example 13. An apparatus according to Example 12, wherein the learning also includes: learning a representation of background traffic in the Underlay network by the first machine learning model, the learning of the representation of the background traffic in the Underlay network using minimization of the difference between the average delay in the Underlay network and the measured delay between at least one node pair of the Overlay network.

[0108] Example 14. An apparatus according to any one of Examples 12 to 13, wherein the output further comprises: outputting the representation of the background service in the underlay network.

[0109] Example 15. The apparatus according to any one of Examples 12 to 14 further includes: learning one or more routing parameters for the overlay network by a second machine model, wherein the learning uses at least the representation of the underlay network and / or the representation of background services in the underlay network.

[0110] Example 16: The apparatus according to any one of Examples 12 to 15 further includes: receiving at least a second service matrix for the overlay network as a first input to a second machine learning model.

[0111] Example 17: The apparatus of any one of Examples 12 to 16, further comprising: receiving the representation of the Underlay network as a second input to the second machine learning model.

[0112] Example 18: The apparatus according to any one of Examples 12 to 17 further includes: receiving a representation of background traffic in the underlay network as a third input to the second machine learning model.

[0113] Example 19: The apparatus according to any one of Examples 12 to 18 further includes: the second machine learning model learning a representation of one or more routing parameters for the overlay network by minimizing the average delay of node pairs of the overlay network.

[0114] Example 20: The apparatus according to any one of Examples 12 to 19 further includes: the second machine learning model outputting the representation of the one or more routes for the overlay network to implement routing configuration in the overlay network.

[0115] Example 21: An apparatus according to any one of Examples 12 to 20, wherein the second machine learning model includes: an input layer that receives the following items: the second service matrix for the Overlay network, the representation of the Underlay network, and the representation of the background service in the Underlay network, an intermediate layer that determines the following items: one or more intermediate values ​​for one or more segment flows, one or more link flows, one or more link delays, and one or more calculated delays, and an output layer that outputs the following items to implement configuration of routing in the Overlay network: the representation of the one or more routing parameters for the Overlay network.

[0116] Example 22: An apparatus according to any one of Examples 12 to 21, wherein the first machine learning model comprises: an input layer that receives the following items: the first input, the second input, and the third input, an intermediate layer that determines the following items: one or more intermediate values ​​for one or more segmented flows, one or more link delays, and one or more calculated delays, and an output layer that outputs the following items: the representation of the Underlay network.

[0117] Example 23: A non-transitory computer-readable storage medium, comprising instructions that, when executed by at least one processor, cause operations, including: receiving at least a first traffic matrix as a first input to a first machine learning model, the first traffic matrix indicating the amount of traffic routed between at least one node pair of an overlay network; receiving information indicating the following as a second input to the first machine learning model: Overlay network routing between the at least one node pair of the overlay network; receiving as a third input to the first machine learning model the measured delay between the at least one node pair of the overlay network; learning a representation of an underlay network by the first machine learning model, the learning using minimization of the difference between an average delay in the underlay network and the measured delay between the at least one node pair of the overlay network; and outputting the representation of the underlay network by the first machine learning model.

[0118] The subject matter described herein may be embodied in a system, apparatus, method and / or article depending on the desired configuration. For example, the base station and user equipment (or one or more components thereof) and / or the process described herein may be implemented using one or more of the following: a processor that executes program code, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an embedded processor, a field programmable gate array (FPGA), and / or a combination thereof. These various implementations may include implementations in one or more computer programs that may be executed and / or interpreted on a programmable system that includes at least one programmable processor that may be dedicated or general-purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to send data and instructions thereto. These computer programs (also referred to as programs, software, software applications, applications, components, program codes, or codes) include machine instructions for programmable processors and may be implemented in high-level procedures and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the term "computer-readable medium" refers to any computer program product, machine-readable medium, computer-readable storage medium, apparatus and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions. Similarly, systems that may include a processor and a memory coupled to the processor are also described herein. The memory may include one or more programs that cause the processor to perform one or more operations described herein.

[0119] Although some variations are described in detail above, other modifications or additions are possible. Specifically, in addition to the features and / or variations described herein, additional features and / or variations may be provided. In addition, the above implementations may be directed to various combinations and subcombinations of the disclosed features and / or combinations and subcombinations of several further features disclosed above. Other embodiments may be within the scope of the following claims.

[0120] If desired, the different functions discussed herein may be performed in a different order and / or concurrently with each other. In addition, if desired, one or more of the above functions may be optional or may be combined. Although various aspects of some embodiments are set forth in the independent claims, other aspects of some embodiments include other combinations of features of the embodiments and / or dependent claims with features of the independent claims, rather than just the combinations explicitly set forth in the claims. It is also noted herein that although example embodiments are described above, these descriptions should not be considered restrictive. On the contrary, several variations and modifications may be made without departing from the scope of some embodiments defined in the appended claims. Other embodiments may be within the scope of the following claims. The term "based on" includes "based on at least". Unless otherwise stated, the use of the phrase "such as" means "such as, for example".

Claims

1. A method for communication, comprising: Receiving at least a first traffic matrix as a first input to a first machine learning model, the first traffic matrix indicating a volume of traffic routed between at least one node pair of an overlay network; receiving, as a second input to the first machine learning model, information indicative of: an overlay network route between the at least one pair of nodes of the overlay network; Receiving a measured delay between at least one pair of nodes of the overlay network as a third input to the first machine learning model; Learning a representation of an underlay network by the first machine learning model, the learning using minimization of a difference between an average delay in the underlay network and the measured delay between the at least one node pair of the overlay network; as well as The representation of the Underlay network is output by the first machine learning model.

2. The method according to claim 1, wherein the learning further comprises: The representation of the background traffic in the Underlay network is learned by the first machine learning model, and the learning of the representation of the background traffic in the Underlay network uses the minimization of the difference between the average delay in the Underlay network and the measured delay between the at least one node pair of the Overlay network.

3. The method according to claim 2, wherein the output further comprises: The representation of the background traffic in the underlay network is outputted.

4. The method according to claim 1, further comprising: One or more routing parameters for the overlay network are learned by a second machine model, and the learning uses at least the representation of the underlay network and / or the representation of background traffic in the underlay network.

5. The method according to claim 1, further comprising: At least a second traffic matrix for the overlay network is received as a first input to a second machine learning model.

6. The method according to claim 5, further comprising: The representation of the Underlay network is received as a second input to the second machine learning model.

7. The method according to claim 6, further comprising: A representation of background traffic in the underlay network is received as a third input to the second machine learning model.

8. The method according to claim 7, further comprising: The second machine learning model learns representations of one or more routing parameters for the overlay network by minimizing the average delay of pairs of nodes in the overlay network.

9. The method according to claim 8, further comprising: The second machine learning model is used to output the representation of the one or more routes for the overlay network to configure routing in the overlay network.

10. The method of claim 8, wherein the second machine learning model comprises: receiving an input layer of: the second service matrix for the overlay network, the representation of the underlay network, and the representation of background services in the underlay network, determining an intermediate level of: one or more intermediate values ​​for one or more segment flows, one or more link flows, one or more link delays, and one or more calculated delays, and An output layer that outputs the following items to implement configuring routing in the overlay network: the representation of the one or more routing parameters for the overlay network.

11. The method of claim 1, wherein the first machine learning model comprises: an input layer receiving: the first input, the second input, and the third input, determining an intermediate level of: one or more intermediate values ​​for one or more segment flows, one or more link delays, and one or more calculated delays, and An output layer that outputs: the representation of the Underlay network.

12. An apparatus for communication, comprising: at least one processor; as well as at least one memory, the at least one memory comprising instructions that, when executed by the at least one processor, cause operations comprising: Receiving at least a first traffic matrix as a first input to a first machine learning model, the first traffic matrix indicating a volume of traffic routed between at least one node pair of an overlay network; receiving, as a second input to the first machine learning model, information indicative of: an overlay network route between the at least one pair of nodes of the overlay network; Receiving a measured delay between at least one pair of nodes of the overlay network as a third input to the first machine learning model; Learning a representation of an underlay network by the first machine learning model, the learning using minimization of a difference between an average delay in the underlay network and the measured delay between the at least one node pair of the overlay network; and The representation of the Underlay network is output by the first machine learning model.

13. The apparatus according to claim 12, wherein the learning further comprises: The representation of the background traffic in the Underlay network is learned by the first machine learning model, and the learning of the representation of the background traffic in the Underlay network uses minimization of the difference between the average delay in the Underlay network and the measured delay between the at least one node pair of the Overlay network.

14. The apparatus of claim 13, wherein the output further comprises: The representation of the background traffic in the underlay network is outputted.

15. The apparatus according to claim 12, further comprising: One or more routing parameters for the overlay network are learned by a second machine model, and the learning uses at least the representation of the underlay network and / or the representation of background traffic in the underlay network.

Images