Methods and systems for multiple label distribution protocol (LDP) forwarding equivalence class (FEC) withdrawals based on an enhanced label mapping message

Abstract

Embodiments include methods, electronic devices, storage medium, and computer program for multiple Label Distribution Protocol (LDP) Forwarding Equivalence Class (FEC) withdrawals based on an enhanced label mapping message. In one embodiment, a method comprises: determining that a LDP session for receiving packets of a traffic flow by a first router from a second router has failed based on a detection by the second router, the packets of the traffic flow having been forwarded to reach the first router as an egress router of the traffic flow; and transmitting, by the second router, a label withdraw message to a third router from which the second router receives packets of the traffic flow, the label withdraw message including a first type-length-value (TLV) that identifies the first router and that is to cause the third router to withdraw labels for a plurality of FECs that map to the first router.

Description

METHODS AND SYSTEMS FOR MULTIPLE LABEL DISTRIBUTION PROTOCOL (LDP) FORWARDING EQUIVALENCE CLASS (FEC) WITHDRAWALS BASED ON AN ENHANCED LABEL MAPPING MESSAGE TECHNICAL FIELD

[0001] Embodiments of the invention relate to the field of networking; and more specifically, to multiple Label Distribution Protocol (LDP) Forwarding Equivalence Class (FEC) withdrawals based on an enhanced label mapping message.BACKGROUND ART

[0002] Label Distribution Protocol (LDP) is a fundamental protocol in Multiprotocol Label Switching (MPLS) networks. A Label Switch Router (LSR) is a router that implements LDP and that forwards packets based on MPLS labels (also referred to as labels and the two terms are used interchangeably herein) rather than traditional Internet Protocol (IP) addresses or other packet features. This label-based forwarding makes packet handling faster and more efficient, especially in large, high-performance networks.

[0003] In LDP, an ingress LSR of a traffic flow is the first LSR in a Label Switched Path (LSP) that receives an IP packet of the traffic flow and imposes an MPLS label before forwarding it to the next-hop LSR, which may be a transit LSR of the traffic flow. One or more transit LSRs may forward the packet toward the egress LSR of the traffic flow based on the label received. The egress LSR of the traffic flow is the final router in the LSP that pops the label of the packet and may forward the packet based on IP routing.

[0004] The egress LSR may assign the original label to a destination prefix, where the label is referred to as a downstream-assigned label since LDP distributes labels from the downstream egress LSR, through transit LSRs, to the ingress LSR, which then imposes labels to packets. That is, label assignments in LDP are performed in the direction from a downstream LSR to an upstream LSR, while packet forwarding is performed in the reverse direction, and the upstream LSR uses labels based on the label assignment received from downstream to encapsulate packets and forward the resulting packets.

[0005] A downstream LSR is the router that is closer to the packet’s destination and an upstream LSR is closer to the packet’s source. The former is responsible for generating and advertising labels for a specific Forwarding Equivalence Class (FEC); and the latter, based on the advertisement, knows which label to use for a given FEC, and optionally may assign its own label for the given FEC.

[0006] An FEC in LDP groups packets that should be treated the same way, meaning they follow the same LSP. Each FEC is associated with a label, and routers use these labels to forward packets without repeatedly inspecting their IP headers. The process of as sociating / as signing a label to an FEC is referred to as label binding.

[0007] When an LDP session fails, the upstream LSRs withdraw all labels associated with the downstream LSR of the failed LDP session provided these labels are not advertised by another downstream LSR, so that the packets won’t be forwarded toward the failed LSR. The label withdrawals are done by sending multiple label withdraw messages, each for an FEC. These multiple label withdraw messages for the same LSR / LDP session failure, consume significant resources in the network, including network bandwidth, execution resources, and / or storage space.SUMMARY OF THE INVENTION

[0008] Embodiments include methods, network device, storage medium, and computer program for multiple Label Distribution Protocol (LDP) Forwarding Equivalence Class (FEC) withdrawals based on an enhanced label mapping message. In one embodiment, a method comprises: determining that a Label Distribution Protocol (LDP) session for receiving packets of a traffic flow by a first router from a second router has failed based on a detection by the second router, the packets of the traffic flow having been forwarded to reach the first router as an egress router of the traffic flow; and transmitting, by the second router, a label withdraw message to a third router from which the second router receives packets of the traffic flow, the label withdraw message including a first type-length- value (TLV) that identifies the first router and that is to cause the third router to withdraw labels for a plurality of Forwarding Equivalence Classes (FECs) that map to the first router.

[0009] Embodiments include network devices to withdraw multiple FECs based on an enhanced label mapping message. In one embodiment, a network device comprising a processor and non-transitory machine -readable storage medium that provides instructions that, when executed by the processor, are capable of causing the processor to perform: determining that a Label Distribution Protocol (LDP) session for receiving packets of a traffic flow by a first other network device from the network device has failed, the packets of the traffic flow having been forwarded to reach the first other network device as an egress other network device of the traffic flow; and transmitting a label withdraw message to a second other network device from which the network device receives packets of the traffic flow, the label withdraw message including a first type-length-value (TLV) that identifies the first other network device and that is to cause the second other network device to withdraw labels for aplurality of Forwarding Equivalence Classes (FECs) that map to the first other network device.

[0010] Embodiments include machine-readable storage media for segment routing path selection at a headend based on machine learning. In one embodiment, a machine-readable storage medium provides instructions that, when executed by a processor, are capable of causing the processor to perform: determining that a Label Distribution Protocol (LDP) session for receiving packets of a traffic flow by a first router from a second router has failed based on a detection by the second router, the packets of the traffic flow having been forwarded to reach the first router as an egress router of the traffic flow; and transmitting, by the second router, a label withdraw message to a third router from which the second router receives packets of the traffic flow, the label withdraw message including a first type-lengthvalue (TLV) that identifies the first router and that is to cause the third router to withdraw labels for a plurality of Forwarding Equivalence Classes (FECs) that map to the first router.

[0011] Through these embodiments, a label mapping message includes an additional TLV to identify the egress router, and when an LDP session fails, the upstream routers may withdraw multiple FECs upon receiving a single label withdraw message. The withdrawal of multiple FECs using the single label withdraw message reduces the consumption of network resources due to the LDP session failure comparing to using multiple label messages each for one FEC.BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The invention may best be understood by referring to the following description and accompanying drawings that are used to illustrate embodiments of the invention. In the drawings:

[0013] Figure 1 illustrates Forwarding Equivalence Class (FEC) withdrawal with one label withdraw message per FEC.

[0014] Figure 2A illustrates using enhanced label mapping messages to indicate the Forwarding Equivalence Class (FEC) originating router per some embodiments.

[0015] Figure 2B illustrates withdrawal of multiple Forwarding Equivalence Classes (FECs) with one label withdraw message per some embodiments.

[0016] Figure 3 illustrates type-length-value (TLVs) for withdrawal of multiple Forwarding Equivalence Classes (FECs) with one label withdraw message per some embodiments.

[0017] Figure 4 is a flow diagram illustrating operations for segment routing path selection based on machine learning per some embodiments.

[0018] Figure 5A illustrates connectivity between network devices (NDs) within an exemplary network, as well as three exemplary implementations of the NDs, according to some embodiments of the invention.

[0019] Figure 5B illustrates an exemplary way to implement a special-purpose network device according to some embodiments of the invention.

[0020] Figure 5C illustrates various exemplary ways in which virtual network elements (VNEs) may be coupled according to some embodiments of the invention.

[0021] Figure 5D illustrates a network with a single network element (NE) on each of the NDs, and within this straightforward approach contrasts a traditional distributed approach (commonly used by traditional routers) with a centralized approach for maintaining reachability and forwarding information (also called network control), according to some embodiments of the invention.

[0022] Figure 5E illustrates the simple case of where each of the NDs implements a single NE, but a centralized control plane has abstracted multiple of the NEs in different NDs into (to represent) a single NE in one of the virtual network(s), according to some embodiments of the invention.

[0023] Figure 5F illustrates a case where multiple VNEs are implemented on different NDs and are coupled to each other, and where a centralized control plane has abstracted these multiple VNEs such that they appear as a single VNE within one of the virtual networks, according to some embodiments of the invention.

[0024] Figure 6 illustrates an example of a communication system per some embodiments. DETAILED DESCRIPTION

[0025] Generally, all terms used herein are to be interpreted according to their ordinary meaning in the relevant technical field, unless a different meaning is clearly given and / or is implied from the context in which it is used. All references to a / an / the element, apparatus, component, means, step, etc. are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any methods disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as following or preceding another step and / or where it is implicit that a step must follow or precede another step. Any feature of any of the embodiments disclosed herein may be applied to any other embodiment, wherever appropriate. Likewise, any advantage of any of the embodiments may apply to any otherembodiments, and vice versa. Other objectives, features, and advantages of the enclosed embodiments will be apparent from the following description.Label Distribution Protocol (LDP) Label Withdrawal

[0026] LDP performs label binding to associate / assign MPLS labels to respective Forwarding Equivalence Class (FECs), and the labels allow the network to forward packets efficiently in an MPLS network without relying on IP addresses or other attributes of the packets. When an LDP session is failed or degraded, and the upstream node of the LDP session can no longer forward packets to the downstream node of the LDP session, the upstream node then notifies nodes of its upstream to withdraw all the affected FECs.

[0027] Figure 1 illustrates Forwarding Equivalence Class (FEC) withdrawal with one label withdraw message per FEC. Four routers form an MPLS network as shown in the figure. While many traffic flows may be forwarded within the MPLS network through LDP, traffic flow 120 is used to illustrate the FEC withdrawal operations. While processes relating to the label withdrawal are discussed herein, more details relating to LDP may be found elsewhere as LDP is standardized in standards such as Internet Engineering Task Force (IETF) Request for Comments (RFC) 5036, entitled “LDP Specification” and dated October 2007.

[0028] Traffic flow 120 enters the MPLS network at router 109, which is referred to as an ingress router since router 109 is the entry point of traffic flow 120. An ingress router may also be referred to as a source router, an entry router, an initial router, a traffic entry point, or similar terms. A traffic flow may be defined as a set of packets whose headers match a given pattern of bits. A flow may be identified by a set of attributes embedded to one or more packets of the flow. An exemplary set of attributes includes a 5 -tuple (source and destination IP addresses, a protocol type, source and destination transmission control protocol or User Datagram Protocol (TCP / UDP) ports.

[0029] Packets of traffic flow 120 are passed through routers 104 and 106, which are referred to as the traffic flow’s transit routers one and two (TRI and TR2), respectively, for they forward MPLS-labeled packets based on label switching rules and do not originate or terminate traffic flow 120. Routers 104 and 106 pass on (optionally swaps labels as packets move) packets of traffic flow 120 through the MPLS network. A transit router is also referred to as an intermediate router, forwarding router, hop router, or similar terms.

[0030] Packets of traffic flow 120 exit the MPLS network at router 102, which is referred to as the egress router, for router 102 is the final destination for the MPLS -labeled packets. Egress router 102 may remove the MPLS label of a packet (e.g., through label popping) andmay forward the packet further, e.g., to another network using traditional IP routing. An egress router is also referred to as an exit router, a destination router, a final router, or similar terms.

[0031] Packets of traffic flow 120 are thus forwarded through the MPLS network. The path in the MPLS for traffic flow 120 is referred to as a label switched path (LSP) for the traffic flow. Prior to traffic flow 120 being forwarded, a label distribution process is performed on the LSP, where label mapping messages are used to distribute labels along the path between egress router (ER) 102 and ingress router (IR) 109. The original label binding for traffic flow 120 is performed by ER 102. ER 102 determines the one or more Forwarding Equivalence Classes (FECs) that it needs to bind with labels. An FEC may be associated with an IP prefix, a virtual private network (VPN) route, a Layer 2 service, a host, or another feature relating to a traffic flow. ER 102 may select an FEC (e.g., destination prefix) and assign a label to it, then send this label to LDP neighbors upstream to TRI 104, which receives the label and uses it to forward packets of traffic flow 120. TRI 104 may replace the label with another label for the FEC. The label distribution is performed between routers using a label mapping message.

[0032] Note that these routers may be referred to with other names such as label switched routers (LSRs), network devices, network nodes, or similar terms. For the discussion relating to label switching herein, these terms are used interchangeably.

[0033] To forward packets of traffic flow 120, LDP sessions are established between peer routers to exchange label mappings. These LDP sessions allow routers to dynamically assign and distribute labels to facilitate label-switched paths (LSPs) for traffic forwarding, reducing the need for manual configuration. For example, LDP sessions are established between ER 102 and TRI 104, TRI 104 and TR2 106, and between TR2 106 and IR 109 in the LSP.

[0034] An LDP session may fail for various reasons. For example, the LDP session may fail due to hardware failure / degradation of the peer routers of the LDP session, physical link failure / degradation between two LDP neighbors, and / or L2 frame or L3 IP route unreachability. The LDP session failure may be detected, by the upstream peer of the LDP session (e.g., TRI 104 in the LDP session between ER 102 and TRI 104) based on timeout of keepalive messages, Transmission Control Protocol (TCP) connection failures, LDP notification messages, Interior Gateway Protocol (IGP) adjacency failure, Bidirectional Forwarding Detection (BFD), or other means.

[0035] When a session failure is detected in the LDP session between ER 102 and TRI 104, packets of traffic flow 120 can no longer be transmitted from TRI 104 to ER 102. TRI 104 removes label bindings received from ER 102, and marks corresponding FECs as unreachable, and withdraw the affected labels from its label forwarding information base (LFIB).

[0036] TRI 104 also notifies its upstream routers for traffic flow 120 so the upstream routers, TR2 106 and IR 109 in this example, no longer forward packets of traffic flow 120 toward the failed LDP session, since such forwarding will result in packet drop and waste resources of the MPLS network.

[0037] The notification to upstream routers for a failed downstream LDP session is performed using label withdraw messages, each for an FEC originating from ER 102, as ER 102 is the egress route that is no longer accessible. ER 102 may have performed label bindings for multiple FECs, and the upstream routers (e.g., TR2 106 and IR 106) that are not part of the failed LDP session need to withdraw the FECs originating from ER 102 and using the failed LDP session. As shown in the figure, a first label withdraw message for FEC 1, and a nthlabel withdraw message for FEC n are sent from TRI 104 upstream. Upon receiving the label withdraw message for the FEC, an upstream router performs similar operations as the ones by TRI 104, to remove label binding for an FEC, marking the FEC unreachable, and withdraw the corresponding label from its LFIB.

[0038] These multiple label withdraw messages for a single failed LDP session are thus broadcasted upstream, when the downstream egress router of the LDP session is no longer reachable from an upstream router (the upstream peer router 104 and the other upstream routers further away from the egress router) on the LSP. Each upstream router, upon receiving each of these label withdraw messages, will remove one FEC originating from the egress router indicated in the corresponding label withdraw message.

[0039] The broadcast of multiple label withdraw message for a single failed LDP session is inefficient, as all label withdraw messages take bandwidth to forward, memory to store, and executing units (e.g., central processing units (CPUs) and / or graphics processing units (GPUs) to process. Additionally, using multiple label withdraw messages to withdraw FECs originating from the same originator router may take a long time to converge. It is thus preferable if a single label withdraw message may be sent for the single failed LDP session, so that the receiving upstream router may withdraw all of the FECs originating from the egress router.

[0040] LDP provides a wildcard FEC in standards such as IETF RFC 5036 (mentioned herein above) and RFC 5918, entitled “entitled “Label Distribution Protocol (LDP) ‘Typed Wildcard’ Forwarding Equivalence Class (FEC)” and dated January 2020. The wildcard FEC may be applied to all the FECs of a router (FEC type = 0) or certain FEC types of the router (e.g., as specified in RFC 5918). Yet existing approaches do not use wildcard FEC to associate with a particular FEC originator.Enhanced label mapping message and label withdraw message

[0041] Figure 2A illustrates using enhanced label mapping messages to indicate the Forwarding Equivalence Class (FEC) originating router per some embodiments. Figure 2A shows the same four-router MPLS network as in Figure 1, but prior to any LDP session failure and the routers are enhanced to process enhanced label mapping messages and label withdraw messages.

[0042] The same traffic flow 120 is assumed to be forwarded through the LSP shown in Figure 1. The label mapping messages for the label binding originating from egress router 102, however, are enhanced to indicate egress router 102 as the originating router of the corresponding FECs, when the upstream routers advertise using label mapping messages for traffic flow 120. Reference 152 shows the enhanced label mapping message indicating the FEC originating router, which is ER 102 in this example. The indication of the FEC originating router is an identifier (ID) of the FEC originating router in some embodiments.

[0043] An enhanced label mapping message is first sent from ER 102 to TRI 104, which then stores the indication of the FEC originating router with the FEC. TRI 104 may send an enhanced label mapping message upstream for the FEC, with the indication of the FEC originating router as well, so that TR2 106 stores the indication of the FEC originating router with the FEC based on the received label mapping message. The process continues at all upstream routers of traffic flow 120 in the label binding process for the FEC.

[0044] The enhanced label mapping messages may be used for all the FECs originating from ER 102. Through the process, all the upstream routers store these FECs with the indication of ER 102 (e.g., the router ID of ER 102). For example, the FEC-to-label mappings and the indication of ER 102 are then stored in a database coupled to an upstream router, e.g., a label information base (LIB). The database maintains a mapping of the FEC, the assigned label, and the FEC originating router. Through the process, each upstream router for traffic flow 120 will maintain the mappings for all the FECs originating from ER 102.

[0045] Figure 2B illustrates withdrawal of multiple Forwarding Equivalence Classes (FECs) with one label withdraw message per some embodiments. Figure 2B is similar to Figure 2A, but now with the LDP session failure between ER 102 and TR 104. Upon the LDP session failure being detected, TR 104 as the upstream router of the LDP session, removes label bindings received from ER 102, and marks corresponding FECs as unreachable, and withdraws the affected labels from its LFIB, similar to the process discussed in Figure 1.

[0046] Yet for notifying its upstream routers, TRI 104 uses an enhanced label withdraw message as shown at reference 154. The enhanced label withdraw message includes anindication of the FEC originating router, ER 102 in this example. Additionally, the enhanced label withdraw message includes a wildcard FEC, e.g., one specified by RFC 5036 and / or RFC 5918, so that all FECs assigned by the FEC originating router, ER 102, may be withdrawn at an upstream router based on the enhanced label withdraw message. The enhanced label withdraw message is advertised upstream to reach all upstream routers for traffic flow 120, and each upstream router may withdraw all FECs originating from ER 102 based on a single enhanced label withdraw message.

[0047] The single enhanced label withdraw message thus replaces all individual label withdraw message per FEC, and each upstream router may perform the FEC withdraw more efficiently without consuming all the resources needed to forward, store, and process the individual label withdraw message. In addition to the efficiency of resource consumption, the LSP route update may be converged more quickly as all the FEC withdrawals may be performed concurrently at an upstream router.

[0048] To use the enhanced label mapping messages and label withdraw messages as disclosed, routers may be configured with the capability. If all the routers in an MPLS network are configured with the capability to process the enhanced label mapping messages and label withdraw messages, a single label withdraw message may then withdraw all the FECs originating from the same originating router. When some routers do not support the feature, the feature may be disabled to ensure backward computability.

[0049] The enhanced label mapping messages and label withdraw messages in the disclosed embodiments introduce changes of existing LDP messages hence may introduce incompatibility with routers not supporting the feature. Yet this trade-off is justified by the significant operational efficiency it enables. The per-FEC signaling in the existing approach for withdrawals becomes a bottleneck in large-scale networks with thousands of FECs.

[0050] In contrast, the disclosed embodiments consolidate these withdrawals using a wildcard FEC with an indication of the FEC originating router, reducing the complexity from O(N) to 0(1). Thus, rather than handling thousands of individual FEC withdrawals, a single message can achieve the same effect, drastically reducing control-plane overhead and improving convergence time during failure events. This improvement is particularly critical for maintaining strict service level agreements (SLAs) in service provider networks, where even milliseconds of delay in route convergence can impact end-user services.

[0051] Furthermore, modem routers are well-equipped with sufficient processing power and memory to handle the additional field for the indication of the FEC originating router without noticeable performance degradation. The incremental nature of this enhancementensures backward compatibility, where allowing gradual deployment without requiring a disruptive, network-wide migration. Given that LDP is widely deployed, the benefits of reducing FEC withdrawal overhead far outweigh the minor cost of including additional information in LDP messages.

[0052] Note that the label mapping / withdraw message enhancement as disclosed do not conflict with other LDP enhancement. For example, a LDP graceful restart mechanism is covered in RFC 3478, entitled “Graceful Restart Mechanism for Label Distribution Protocol” and dated February 2003. Normally, if an LDP session fails, MPLS forwarding breaks, causing packet loss. RFC 3478 defines a graceful restart that enables routers to retain label bindings temporarily while the control plane is restored, avoiding traffic drops.

[0053] The label mapping / withdraw message enhancement as disclosed complements the grace restart mechanism. If a router restarts and successfully retains its FEC state, no FEC withdrawal is necessary. However, if the failure is severe enough that the router does not recover, the wildcard FEC withdrawal mechanism disclosed herein ensures that upstream routers do not continue forwarding based on outdated FEC mappings. This dual-layered approach enhances the overall robustness of LDP-based networks, addressing both temporary and permanent failures more effectively than existing mechanisms.Exemplary TLVs

[0054] Embodiments disclosed herein include enhanced label mapping messages and label withdraw messages. Exemplary TLVs to be included in these messages are explained herein.Figure 3 illustrates type-length-value (TLVs) for withdrawal of multiple Forwarding Equivalence Classes (FECs) with one label withdraw message per some embodiments.

[0055] At reference 320, an enhanced label mapping message is shown, which includes a message header 301 and multiple TLVs. The message header 301 includes information such as LDP protocol version, Protocol Data Unit (PDU) length, and LDP identifier (ID). The LPD ID uniquely identifies a router and may include a router ID and an instance ID to distinguish multiple LDP instances in the router in some embodiments.

[0056] The message type TLV 302 indicates that the message type is label mapping. The FEC TLV 304 specifies the FEC for which the label is being advertised. The label TLV 306 contains the label assigned to the FEC. A label mapping message may include optional TLVs 308 that proprietary TLVs may be implemented. In some embodiments, an originator LSR ID sub-TLV 310 is included in the optional TLVs 308.

[0057] The originator LSR ID sub-TLV 310 includes a sub-TLV type field 312 (e.g., two bytes long) to indicate the sub-TLV is for identifying the FEC originating router, a sub-TLVlength field 314 (e.g., four bytes) to indicate the length of the sub-TLV, and a router ID field 316 to identify the FEC originating router, which is done using the router ID, or router ID + LDP instance ID in some embodiments.

[0058] At reference 370, an enhanced label withdraw message is shown. The enhanced label withdraw message includes a message header 361 that is similar to the message header 301. The included message type TLV 362 indicates that the message type being label withdrawing. The FEC TLV 364 indicates multiple FECs sourced from an FEC originating router. While the multiple FECs may be enumerated in the FEC TLV 364 in some embodiments, in alternative embodiments, the multiple FECs sourced from the FEC originating router are represented with a wildcard FEC as discussed herein. The label TLV 366 is set to an implicit NULL value in some embodiments because the NULL value applies to all FECs. The label withdraw message may include optional TLVs 308 that include the originator LSR ID sub-TLV 310.Operations per some embodiments

[0059] Figure 4 is a flow diagram illustrating operations for withdrawal of multiple Forwarding Equivalence Classes (FECs) with one label withdraw message per some embodiments. The operations of method 400 may be implemented on a router (e.g., TRI 104 or network devices in Figures 5A to 5F).

[0060] At reference 402, it is determined that a Label Distribution Protocol (LDP) session for receiving packets of a traffic flow by a first router from a second router has failed based on a detection by the second router, the packets of the traffic flow have been forwarded to reach the first router as an egress router of the traffic flow. For example, the first router is ER 102, and the second router is TRI 104 discussed herein. The detection may be based on timeout of keepalive messages, TCP connection failures, LDP notification messages, IGP adjacency failure, BFD, or other means.

[0061] At reference 404, the second router transmits a label withdraw message to a third router from which the second router receives packets of the traffic flow, the label withdraw message includes a first type-length-value (TLV) that identifies the first router and that is to cause the third router to withdraw labels for a plurality of Forwarding Equivalence Classes (FECs) that map to the first router. For example, the third router is TR2 106 discussed herein.

[0062] In some embodiments, the first TLV is a sub-TLV that indicates an identifier of the first router to identify the first router.

[0063] In some embodiments, the sub-TLV includes a type field indicating the sub-TLV is for identifying the first router, a length field indicating length of the sub-TLV, and a valuefield indicating the identifier of the first router. In some embodiments, the type field is two bytes (16 bits) long, the length field is four type (32 bits) long, and the value field is four bytes (32 bits) long.

[0064] In some embodiments, the first TLV is generated based on a second TLV that is included in a first label mapping message transmitted by the first router and propagated from the first router to remaining routers through which the traffic flow is forwarded. The first label mapping message is discussed relating to Figures 2B and 3.

[0065] In some embodiments, propagating the second TLV comprises: upon a router receiving the first label mapping message, the router stores an identifier of the first router, one or more FECs mapped to the first router, and corresponding labels.

[0066] In some embodiments, propagating the second TLV further comprises transmitting a second label mapping message from the router to another router that is further away to the first router in a label switched path including the LDP session for the traffic flow, the second label mapping message including the second TLV.

[0067] In some embodiments, the plurality of FECs are represented by a single wildcard FEC.

[0068] In some embodiments, withdrawing the labels for the plurality of FECs comprises removing the plurality of FECs in a label forwarding table (LFIB) of the third router.

[0069] In some embodiments, failure of the LDP session is due to degradation of links or routers involved in the LDP session.

[0070] Through these embodiments, a label mapping message includes an additional TLV to identify the egress router, and when an LDP session fails, the upstream routers may withdraw multiple FECs upon receiving a single label withdraw message. The withdrawal of multiple FECs by a single label withdraw message reduces the consumption of network resources due to the LDP session failure.Devices and Environments for Implementing Embodiments of the Invention

[0071] A network device (ND) is an electronic device that communicatively interconnects other electronic devices on the network (e.g., other network devices, end-user devices). Some network devices are “multiple services network devices” that provide support for multiple networking functions (e.g., routing, bridging, switching, Layer 2 aggregation, session border control, Quality of Service, and / or subscriber management), and / or provide support for multiple application services (e.g., data, voice, and video). Each of routers 102 to 109 discussed herein may be a network device in some embodiments.

[0072] Figure 5A illustrates connectivity between network devices (NDs) within an exemplary network, as well as three exemplary implementations of the NDs, according to some embodiments of the invention. Figure 5A shows NDs 500A-H, and their connectivity by way of lines between 500A-500B, 500B-500C, 500C-500D, 500D-500E, 500E-500F, 500F-500G, and 500A-500G, as well as between 500H and each of 500A, 500C, 500D, and 500G. These NDs are physical devices, and the connectivity between these NDs can be wireless or wired (often referred to as a link). An additional line extending from NDs 500A, 500E, and 500F illustrates that these NDs act as ingress and egress points for the network (and thus, these NDs are sometimes referred to as edge NDs; while the other NDs may be called core NDs).

[0073] Two of the exemplary ND implementations in Figure 5A are: 1) a special-purpose network device 502 that uses custom application-specific integrated-circuits (ASICs) and a special-purpose operating system (OS); and 2) a general -purpose network device 504 that uses common off-the-shelf (COTS) processors and a standard OS.

[0074] The special-purpose network device 502 includes networking hardware 510 comprising a set of one or more processor(s) 512, forwarding resource(s) 514 (which typically include one or more ASICs and / or network processors), and physical network interfaces (NIs) 516 (through which network connections are made, such as those shown by the connectivity between NDs 500A-H), as well as non-transitory machine readable storage media 518 having stored therein networking software 520. During operation, the networking software 520 may be executed by the networking hardware 510 to instantiate a set of one or more networking software instance(s) 522. Each of the networking software instance(s) 522, and that part of the networking hardware 510 that executes that network software instance (be it hardware dedicated to that networking software instance and / or time slices of hardware temporally shared by that networking software instance with others of the networking software instance(s) 522), form a separate virtual network element 530A-R. Each of the virtual network element(s) (VNEs) 530A-R includes a control communication and configuration module 532A-R (sometimes referred to as a local control module or control communication module) and forwarding table(s) 534A-R, such that a given virtual network element (e.g., 530A) includes the control communication and configuration module (e.g., 532A), a set of one or more forwarding table(s) (e.g., 534A), and that portion of the networking hardware 510 that executes the virtual network element (e.g., 530A). In some embodiments, the networking software 520 includes an FEC withdrawal enhancer 525 to perform operations relating to Figures 1 to 4 discussed herein above.

[0075] The special-purpose network device 502 is often physically and / or logically considered to include: 1) a ND control plane 524 (sometimes referred to as a control plane) comprising the processor(s) 512 that execute the control communication and configuration module(s) 532A-R; and 2) a ND forwarding plane 526 (sometimes referred to as a forwarding plane, a data plane, or a media plane) comprising the forwarding resource(s) 514 that utilize the forwarding table(s) 534A-R and the physical NIs 516. By way of example, where the ND is a router (or is implementing routing functionality), the ND control plane 524 (the processor(s) 512 executing the control communication and configuration module(s) 532A-R) is typically responsible for participating in controlling how data (e.g., packets) is to be routed (e.g., the next hop for the data and the outgoing physical NI for that data) and storing that routing information in the forwarding table(s) 534A-R, and the ND forwarding plane 526 is responsible for receiving that data on the physical NIs 516 and forwarding that data out the appropriate ones of the physical NIs 516 based on the forwarding table(s) 534A-R.

[0076] Figure 5B illustrates an exemplary way to implement the special-purpose network device 502 according to some embodiments of the invention. Figure 5B shows a specialpurpose network device including cards 538 (typically hot pluggable). While in some embodiments the cards 538 are of two types (one or more that operate as the ND forwarding plane 526 (sometimes called line cards), and one or more that operate to implement the ND control plane 524 (sometimes called control cards)), alternative embodiments may combine functionality onto a single card and / or include additional card types (e.g., one additional type of card is called a service card, resource card, or multi-application card). A service card can provide specialized processing (e.g., Layer 4 to Layer 7 services (e.g., firewall, Internet Protocol Security (IPsec), Secure Sockets Layer (SSL) / Transport Layer Security (TLS), Intrusion Detection System (IDS), peer-to-peer (P2P), Voice over IP (VoIP) Session Border Controller, Mobile Wireless Gateways (Gateway General Packet Radio Service (GPRS) Support Node (GGSN), Evolved Packet Core (EPC) Gateway)). By way of example, a service card may be used to terminate IPsec tunnels and execute the attendant authentication and encryption algorithms. These cards are coupled together through one or more interconnect mechanisms illustrated as backplane 536 (e.g., a first full mesh coupling the line cards and a second full mesh coupling all of the cards).

[0077] Returning to Figure 5A, the general-purpose network device 504 includes hardware 540 comprising a set of one or more processor(s) 542 (which are often COTS processors) and physical NIs 546, as well as non-transitory machine -readable storage media 548 having stored therein software 550. During operation, the processor(s) 542 execute the software 550 toinstantiate one or more sets of one or more applications 564A-R. While one embodiment does not implement virtualization, alternative embodiments may use different forms of virtualization. For example, in one such alternative embodiment the virtualization layer 554 represents the kernel of an operating system (or a shim executing on a base operating system) that allows for the creation of multiple instances 562A-R called software containers that may each be used to execute one (or more) of the sets of applications 564A-R; where the multiple software containers (also called virtualization engines, virtual private servers, or jails) are user spaces (typically a virtual memory space) that are separate from each other and separate from the kernel space in which the operating system is run; and where the set of applications running in a given user space, unless explicitly allowed, cannot access the memory of the other processes. In another such alternative embodiment the virtualization layer 554 represents a hypervisor (sometimes referred to as a virtual machine monitor (VMM)) or a hypervisor executing on top of a host operating system, and each of the sets of applications 564A-R is run on top of a guest operating system within an instance 562A-R called a virtual machine (which may in some cases be considered a tightly isolated form of software container) that is run on top of the hypervisor - the guest operating system and application may not know they are running on a virtual machine as opposed to running on a “bare metal” host electronic device, or through para-virtualization the operating system and / or application may be aware of the presence of virtualization for optimization purposes. In yet other alternative embodiments, one, some or all of the applications are implemented as unikemel(s), which can be generated by compiling directly with an application only a limited set of libraries (e.g., from a library operating system (LibOS) including drivers / libraries of OS services) that provide the particular OS services needed by the application. As a unikemel can be implemented to run directly on hardware 540, directly on a hypervisor (in which case the unikernel is sometimes described as running within a LibOS virtual machine), or in a software container, embodiments can be implemented fully with unikemels running directly on a hypervisor represented by virtualization layer 554, unikemels running within software containers represented by instances 562A-R, or as a combination of unikemels and the abovedescribed techniques (e.g., unikernels and virtual machines both run directly on a hypervisor, unikernels and sets of applications that are run in different software containers). In some embodiments, the software 550 includes a FEC withdrawal enhancer 555 to perform operations relating to Figures 1 to 4 discussed herein above.

[0078] The instantiation of the one or more sets of one or more applications 564A-R, as well as virtualization if implemented, are collectively referred to as software instance(s) 552.Each set of applications 564A-R, corresponding virtualization construct (e.g., instance 562A-R) if implemented, and that part of the hardware 540 that executes them (be it hardware dedicated to that execution and / or time slices of hardware temporally shared), forms a separate virtual network element(s) 560A-R.

[0079] The virtual network element(s) 560A-R perform similar functionality to the virtual network element(s) 530A-R - e.g., similar to the control communication and configuration module(s) 532A and forwarding table(s) 534A (this virtualization of the hardware 540 is sometimes referred to as network function virtualization (NFV)). Thus, NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which could be located in Data centers, NDs, and customer premise equipment (CPE). While embodiments of the invention are illustrated with each instance 562A-R corresponding to one VNE 560A-R, alternative embodiments may implement this correspondence at a finer level granularity (e.g., line card virtual machines virtualize line cards, control card virtual machine virtualize control cards, etc.); it should be understood that the techniques described herein with reference to a correspondence of instances 562A-R to VNEs also apply to embodiments where such a finer level of granularity and / or unikernels are used.

[0080] In certain embodiments, the virtualization layer 554 includes a virtual switch that provides similar forwarding services as a physical Ethernet switch. Specifically, this virtual switch forwards traffic between instances 562A-R and the physical NI(s) 546, as well as optionally between the instances 562A-R; in addition, this virtual switch may enforce network isolation between the VNEs 560A-R that by policy are not permitted to communicate with each other (e.g., by honoring virtual local area networks (VLANs)).

[0081] The third exemplary ND implementation in Figure 5A is a hybrid network device 506, which includes both custom ASICs / special-purpose OS and COTS processors / standard OS in a single ND or a single card within an ND. In certain embodiments of such a hybrid network device, a platform VM (i.e., a VM that that implements the functionality of the special-purpose network device 502) could provide for para-virtualization to the networking hardware present in the hybrid network device 506.

[0082] Regardless of the above exemplary implementations of an ND, when a single one of multiple VNEs implemented by an ND is being considered (e.g., only one of the VNEs is part of a given virtual network) or where only a single VNE is currently being implemented by an ND, the shortened term network element (NE) is sometimes used to refer to that VNE. Also in all of the above exemplary implementations, each of the VNEs (e.g., VNE(s) 530A-R,VNEs 56OA-R, and those in the hybrid network device 506) receives data on the physical NIs (e.g., 516, 546) and forwards that data out the appropriate ones of the physical NIs (e.g., 516, 546). For example, a VNE implementing IP router functionality forwards IP packets on the basis of some of the IP header information in the IP packet; where IP header information includes source IP address, destination IP address, source port, destination port (where “source port” and “destination port” refer herein to protocol ports, as opposed to physical ports of a ND), transport protocol (e.g., user datagram protocol (UDP), Transmission Control Protocol (TCP), and differentiated services code point (DSCP) values.

[0083] Figure 5C illustrates various exemplary ways in which VNEs may be coupled according to some embodiments of the invention. Figure 5C shows VNEs 570A.1-570A.P (and optionally VNEs 570A.Q-570A.R) implemented in ND 500A and VNE 570H.1 in ND 500H. In Figure 5C, VNEs 570A.1-P are separate from each other in the sense that they can receive packets from outside ND 500A and forward packets outside of ND 500A; VNE 570A.1 is coupled with VNE 570H.1, and thus they communicate packets between their respective NDs; VNE 570A.2-570A.3 may optionally forward packets between themselves without forwarding them outside of the ND 500A; and VNE 570A.P may optionally be the first in a chain of VNEs that includes VNE 570A.Q followed by VNE 570A.R (this is sometimes referred to as dynamic service chaining, where each of the VNEs in the series of VNEs provides a different service - e.g., one or more layer 4-7 network services). While Figure 5C illustrates various exemplary relationships between the VNEs, alternative embodiments may support other relationships (e.g., more / fewer VNEs, more / fewer dynamic service chains, multiple different dynamic service chains with some common VNEs and some different VNEs).

[0084] The NDs of Figure 5A, for example, may form part of the Internet or a private network; and other electronic devices (not shown; such as end user devices including workstations, laptops, netbooks, tablets, palm tops, mobile phones, smartphones, phablets, multimedia phones, Voice Over Internet Protocol (VOIP) phones, terminals, portable media players, GPS units, wearable devices, gaming systems, set-top boxes, Internet enabled household appliances) may be coupled to the network (directly or through other networks such as access networks) to communicate over the network (e.g., the Internet or virtual private networks (VPNs) overlaid on (e.g., tunneled through) the Internet) with each other (directly or through servers) and / or access content and / or services. Such content and / or services are typically provided by one or more servers (not shown) belonging to a service / content provider or one or more end user devices (not shown) participating in a peer-to-peer (P2P) service, andmay include, for example, public webpages (e.g., free content, store fronts, search services), private webpages (e.g., usemame / password accessed webpages providing email services), and / or corporate networks over VPNs. For instance, end user devices may be coupled (e.g., through customer premise equipment coupled to an access network (wired or wirelessly)) to edge NDs, which are coupled (e.g., through one or more core NDs) to other edge NDs, which are coupled to electronic devices acting as servers. However, through compute and storage virtualization, one or more of the electronic devices operating as the NDs in Figure 5A may also host one or more such servers (e.g., in the case of the general purpose network device 504, one or more of the software instances 562A-R may operate as servers; the same would be true for the hybrid network device 506; in the case of the special-purpose network device 502, one or more such servers could also be run on a virtualization layer executed by the processor(s) 512); in which case the servers are said to be co-located with the VNEs of that ND.

[0085] A virtual network is a logical abstraction of a physical network (such as that in Figure 5A) that provides network services (e.g., L2 and / or L3 services). A virtual network can be implemented as an overlay network (sometimes referred to as a network virtualization overlay) that provides network services (e.g., layer 2 (L2, data link layer) and / or layer 3 (L3, network layer) services) over an underlay network (e.g., an L3 network, such as an Internet Protocol (IP) network that uses tunnels (e.g., generic routing encapsulation (GRE), layer 2 tunneling protocol (L2TP), IPSec) to create the overlay network).

[0086] A network virtualization edge (NVE) sits at the edge of the underlay network and participates in implementing the network virtualization; the network-facing side of the NVE uses the underlay network to tunnel frames to and from other NVEs; the outward-facing side of the NVE sends and receives data to and from systems outside the network. A virtual network instance (VNI) is a specific instance of a virtual network on a NVE (e.g., a NE / VNE on an ND, a part of a NE / VNE on a ND where that NE / VNE is divided into multiple VNEs through emulation); one or more VNIs can be instantiated on an NVE (e.g., as different VNEs on an ND). A virtual access point (VAP) is a logical connection point on the NVE for connecting external systems to a virtual network; a VAP can be physical or virtual ports identified through logical interface identifiers (e.g., a VLAN ID).

[0087] Examples of network services include: 1) an Ethernet LAN emulation service (an Ethernet-based multipoint service similar to an Internet Engineering Task Force (IETF) Multiprotocol Label Switching (MPLS) or Ethernet VPN (EVPN) service) in which external systems are interconnected across the network by a LAN environment over the underlaynetwork (e.g., an NVE provides separate L2 VNIs (virtual switching instances) for different such virtual networks, and L3 (e.g., IP / MPLS) tunneling encapsulation across the underlay network); and 2) a virtualized IP forwarding service (similar to IETF IP VPN (e.g., Border Gateway Protocol (BGP) / MPLS IP VPN) from a service definition perspective) in which external systems are interconnected across the network by an L3 environment over the underlay network (e.g., an NVE provides separate L3 VNIs (forwarding and routing instances) for different such virtual networks, and L3 (e.g., IP / MPLS) tunneling encapsulation across the underlay network)). Network services may also include quality of service capabilities (e.g., traffic classification marking, traffic conditioning and scheduling), security capabilities (e.g., filters to protect customer premises from network - originated attacks, to avoid malformed route announcements), and management capabilities (e.g., full detection and processing).

[0088] Fig. 5D illustrates a network with a single network element on each of the NDs of Figure 5A, and within this straight forward approach contrasts a traditional distributed approach (commonly used by traditional routers) with a centralized approach for maintaining reachability and forwarding information (also called network control), according to some embodiments of the invention. Specifically, Figure 5D illustrates network elements (NEs) 570A-H with the same connectivity as the NDs 500A-H of Figure 5A.

[0089] Figure 5D illustrates that the distributed approach 572 distributes responsibility for generating the reachability and forwarding information across the NEs 570A-H; in other words, the process of neighbor discovery and topology discovery is distributed.

[0090] For example, where the special-purpose network device 502 is used, the control communication and configuration module(s) 532A-R of the ND control plane 524 typically include a reachability and forwarding information module to implement one or more routing protocols (e.g., an exterior gateway protocol such as Border Gateway Protocol (BGP), Interior Gateway Protocol(s) (IGP) (e.g., Open Shortest Path First (OSPF), Intermediate System to Intermediate System (IS-IS), Routing Information Protocol (RIP), Label Distribution Protocol (LDP), Resource Reservation Protocol (RSVP) (including RSVP-Traffic Engineering (TE): Extensions to RSVP for LSP Tunnels and Generalized Multi-Protocol Label Switching (GMPLS) Signaling RSVP-TE)) that communicate with other NEs to exchange routes, and then selects those routes based on one or more routing metrics. Thus, the NEs 570A-H (e.g., the processor(s) 512 executing the control communication and configuration module(s) 532A-R) perform their responsibility for participating in controlling how data (e.g., packets) is to be routed (e.g., the next hop for the data and the outgoing physical NI for that data) bydistributively determining the reachability within the network and calculating their respective forwarding information. Routes and adjacencies are stored in one or more routing structures (e.g., Routing Information Base (RIB), Label Information Base (LIB), one or more adjacency structures) on the ND control plane 524. The ND control plane 524 programs the ND forwarding plane 526 with information (e.g., adjacency and route information) based on the routing structure(s). For example, the ND control plane 524 programs the adjacency and route information into one or more forwarding table(s) 534A-R (e.g., Forwarding Information Base (FIB), Label Forwarding Information Base (LFIB), and one or more adjacency structures) on the ND forwarding plane 526. For layer 2 forwarding, the ND can store one or more bridging tables that are used to forward data based on the layer 2 information in that data. While the above example uses the special-purpose network device 502, the same distributed approach 572 can be implemented on the general purpose network device 504 and the hybrid network device 506.

[0091] Figure 5D illustrates that a centralized approach 574 (also known as software defined networking (SDN)) that decouples the system that makes decisions about where traffic is sent from the underlying systems that forwards traffic to the selected destination. The illustrated centralized approach 574 has the responsibility for the generation of reachability and forwarding information in a centralized control plane 576 (sometimes referred to as a SDN control module, controller, network controller, OpenFlow controller, SDN controller, control plane node, network virtualization authority, or management control entity), and thus the process of neighbor discovery and topology discovery is centralized. The centralized control plane 576 has a south bound interface 582 with a data plane 580 (sometime referred to the infrastructure layer, network forwarding plane, or forwarding plane (which should not be confused with a ND forwarding plane)) that includes the NEs 570A-H (sometimes referred to as switches, forwarding elements, data plane elements, or nodes). The centralized control plane 576 includes a network controller 578, which includes a centralized reachability and forwarding information module 579 that determines the reachability within the network and distributes the forwarding information to the NEs 570A-H of the data plane 580 over the south bound interface 582 (which may use the OpenFlow protocol). Thus, the network intelligence is centralized in the centralized control plane 576 executing on electronic devices that are typically separate from the NDs.

[0092] For example, where the special-purpose network device 502 is used in the data plane 580, each of the control communication and configuration module(s) 532A-R of the ND control plane 524 typically include a control agent that provides the VNE side of the southbound interface 582. In this case, the ND control plane 524 (the processor(s) 512 executing the control communication and configuration module(s) 532A-R) performs its responsibility for participating in controlling how data (e.g., packets) is to be routed (e.g., the next hop for the data and the outgoing physical NI for that data) through the control agent communicating with the centralized control plane 576 to receive the forwarding information (and in some cases, the reachability information) from the centralized reachability and forwarding information module 579 (it should be understood that in some embodiments of the invention, the control communication and configuration module(s) 532A-R, in addition to communicating with the centralized control plane 576, may also play some role in determining reachability and / or calculating forwarding information - albeit less so than in the case of a distributed approach; such embodiments are generally considered to fall under the centralized approach 574, but may also be considered a hybrid approach).

[0093] While the above example uses the special-purpose network device 502, the same centralized approach 574 can be implemented with the general purpose network device 504 (e.g., each of the VNE 560A-R performs its responsibility for controlling how data (e.g., packets) is to be routed (e.g., the next hop for the data and the outgoing physical NI for that data) by communicating with the centralized control plane 576 to receive the forwarding information (and in some cases, the reachability information) from the centralized reachability and forwarding information module 579; it should be understood that in some embodiments of the invention, the VNEs 560A-R, in addition to communicating with the centralized control plane 576, may also play some role in determining reachability and / or calculating forwarding information - albeit less so than in the case of a distributed approach) and the hybrid network device 506. In fact, the use of SDN techniques can enhance the NFV techniques typically used in the general purpose network device 504 or hybrid network device 506 implementations as NFV is able to support SDN by providing an infrastructure upon which the SDN software can be run, and NFV and SDN both aim to make use of commodity server hardware and physical switches.

[0094] Figure 5D also shows that the centralized control plane 576 has a north bound interface 584 to an application layer 586, in which resides application(s) 588. The centralized control plane 576 has the ability to form virtual networks 592 (sometimes referred to as a logical forwarding plane, network services, or overlay networks (with the NEs 570A-H of the data plane 580 being the underlay network)) for the application(s) 588. Thus, the centralized control plane 576 maintains a global view of all NDs and configured NEs / VNEs, and it maps the virtual networks to the underlying NDs efficiently (including maintaining these mappingsas the physical network changes either through hardware (ND, link, or ND component) failure, addition, or removal).

[0095] While Figure 5D shows the distributed approach 572 separate from the centralized approach 574, the effort of network control may be distributed differently or the two combined in certain embodiments of the invention. For example: 1) embodiments may generally use the centralized approach (SDN) 574, but have certain functions delegated to the NEs (e.g., the distributed approach may be used to implement one or more of fault monitoring, performance monitoring, protection switching, and primitives for neighbor and / or topology discovery); or 2) embodiments of the invention may perform neighbor discovery and topology discovery via both the centralized control plane and the distributed protocols, and the results compared to raise exceptions where they do not agree. Such embodiments are generally considered to fall under the centralized approach 574, but may also be considered a hybrid approach.

[0096] While Figure 5D illustrates the simple case where each of the NDs 500A-H implements a single NE 570A-H, it should be understood that the network control approaches described with reference to Figure 5D also work for networks where one or more of the NDs 500A-H implement multiple VNEs (e.g., VNEs 530A-R, VNEs 560A-R, those in the hybrid network device 506). Alternatively or in addition, the network controller 578 may also emulate the implementation of multiple VNEs in a single ND. Specifically, instead of (or in addition to) implementing multiple VNEs in a single ND, the network controller 578 may present the implementation of a VNE / NE in a single ND as multiple VNEs in the virtual networks 592 (all in the same one of the virtual network(s) 592, each in different ones of the virtual network(s) 592, or some combination). For example, the network controller 578 may cause an ND to implement a single VNE (a NE) in the underlay network, and then logically divide up the resources of that NE within the centralized control plane 576 to present different VNEs in the virtual network(s) 592 (where these different VNEs in the overlay networks are sharing the resources of the single VNE / NE implementation on the ND in the underlay network).

[0097] On the other hand, Figures 5E and 5F respectively illustrate exemplary abstractions of NEs and VNEs that the network controller 578 may present as part of different ones of the virtual networks 592. Figure 5E illustrates the simple case of where each of the NDs 500A-H implements a single NE 570A-H (see Figure 5D), but the centralized control plane 576 has abstracted multiple of the NEs in different NDs (the NEs 570A-C and G-H) into (to represent) a single NE 5701 in one of the virtual network(s) 592 of Figure 5D, according to someembodiments of the invention. Figure 5E shows that in this virtual network, the NE 5701 is coupled to NE 570D and 570F, which are both still coupled to NE 570E.

[0098] Figure 5F illustrates a case where multiple VNEs (VNE 570A.1 and VNE 570H.1) are implemented on different NDs (ND 500A and ND 500H) and are coupled to each other, and where the centralized control plane 576 has abstracted these multiple VNEs such that they appear as a single VNE 570T within one of the virtual networks 592 of Figure 5D, according to some embodiments of the invention. Thus, the abstraction of a NE or VNE can span multiple NDs.

[0099] While some embodiments of the invention implement the centralized control plane 576 as a single entity (e.g., a single instance of software running on a single electronic device), alternative embodiments may spread the functionality across multiple entities for redundancy and / or scalability purposes (e.g., multiple instances of software running on different electronic devices).A Wireless Network per Some Embodiments

[0100] Figure 6 illustrates an example of a communication system per some embodiments. In the example, the communication system 600 includes a telecommunication network 602 that includes an access network 604, such as a radio access network (RAN), and a core network 606, which includes one or more core network nodes 608. The access network 604 includes one or more access network nodes, such as network nodes 610A and 610B (one or more of which may be generally referred to as network nodes 610), or any other similar 3rdGeneration Partnership Project (3GPP) access node or non-3GPP access point. The network nodes 610 facilitate direct or indirect connection of user equipment (UE), such as by connecting UEs 612A, 612B, 612C, and 612D (one or more of which may be generally referred to as UEs 612) to the core network 606 over one or more wireless connections.

[0101] Example wireless communications over a wireless connection include transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and / or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, the communication system 600 may include any number of wired or wireless networks, network nodes, UEs, and / or any other components or systems that may facilitate or participate in the communication of data and / or signals whether via wired or wireless connections. The communication system 600 may include and / or interface with any type of communication, telecommunication, data, cellular, radio network, and / or other similar type of system.

[0102] UEs 612 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and / or operable to communicate wirelessly with the network nodes 610 and other communication devices. Similarly, the network nodes 610 are arranged, capable, configured, and / or operable to communicate directly or indirectly with the UEs 612 and / or with other network nodes or equipment in the telecommunication network 602 to enable and / or provide network access, such as wireless network access, and / or to perform other functions, such as administration in the telecommunication network 602.

[0103] In the depicted example, the core network 606 connects the network nodes 610 to one or more hosts, such as host 616. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. The core network 606 includes one more core network nodes (e.g., core network node 608) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and / or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 608. Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDE), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0104] The host 616 may be under the ownership or control of a service provider other than an operator or provider of the access network 604 and / or the telecommunication network 602, and may be operated by the service provider or on behalf of the service provider. The host 616 may host a variety of applications to provide one or more service. Examples of such applications include live and pre-recorded audio / video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.

[0105] As a whole, the communication system 600 of Figure 6 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM);Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and / or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and / or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and / or any low-power wide-area network (LPWAN) standards such as LoRa and Sigfox.

[0106] In some examples, the telecommunication network 602 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunication network 602 may support network slicing to provide different logical networks to different devices that are connected to the telecommunication network 602. For example, the telecommunication network 602 may provide Ultra Reliable Low Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and / or Massive Machine Type Communication (mMTC) / Massive loT services to yet further UEs.

[0107] In some examples, the UEs 612 are configured to transmit and / or receive information without direct human interaction. For instance, a UE may be designed to transmit information to the access network 604 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 604. Additionally, a UE may be configured for operating in single- or multiple radio access technology (multi-RAT) or multi-standard mode. For example, a UE may operate with any one or combination of WiFi, NR (New Radio) and LTE, i.e., being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio -Dual Connectivity (EN-DC).

[0108] In the example, the hub 614 communicates with the access network 604 to facilitate indirect communication between one or more UEs (e.g., UE 612C and / or 612D) and network nodes (e.g., network node 610B). In some examples, the hub 614 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs. For example, the hub 614 may be a broadband router enabling access to the core network 606 for the UEs. As another example, hub 614 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may be received from the UEs, network nodes 610, or by executable code, script, process, or other instructions in the hub 614. As another example, hub 614 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or otherprocessing of the data. As another example, hub 614 may be a content source. For example, for a UE that is a virtual reality (VR) headset, display, loudspeaker or other media delivery device, the hub 614 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 614 then provides to the UE either directly, after performing local processing, and / or after adding additional local content. In still another example, hub 614 acts as a proxy server or orchestrator for the UEs, in particular in if one or more of the UEs are low energy loT devices.

[0109] Hub 614 may have a constant / persistent or intermittent connection to the network node 61 OB. Hub 614 may also allow for a different communication scheme and / or schedule between the hub 614 and UEs (e.g., UE 612C and / or 612D), and between the hub 614 and the core network 606. In other examples, hub 614 is connected to the core network 606 and / or one or more UEs via a wired connection. Moreover, hub 614 may be configured to connect to a machine-to-machine (M2M) service provider over the access network 604 and / or to another UE over a direct connection. In some scenarios, UEs may establish a wireless connection with network nodes 610 while still connected via the hub 614 via a wired or wireless connection. In some embodiments, the hub 614 may be a dedicated hub - that is, a hub whose primary function is to route communications to / from the UEs from / to the network node 610B. In other embodiments, the hub 614 may be a non-dedicated hub - that is, a device which is capable of operating to route communications between the UEs and network node 610B, but which is additionally capable of operating as a communication start and / or end point for certain data channels.

[0110] In some embodiments, the network devices 502, 504, or 506 may implement network nodes 610A-B, which then perform operations discussed herein above relating to Figures 1 to 4.Terms

[0111] References in the specification to “one embodiment,” “an embodiment,” “an example embodiment,” and so forth, indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

[0112] The description and claims may use the terms “coupled” and “connected,” along with their derivatives. These terms are not intended as synonyms for each other. “Coupled” is used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, co-operate or interact with each other. “Connected” is used to indicate the establishment of wireless or wireline communication between two or more elements that are coupled with each other. A “set,” as used herein can refer to any whole number of items including one item.

[0113] An electronic device stores and transmits (internally and / or with other electronic devices over a network) code (which is composed of software instructions and which is sometimes referred to as a computer program code or a computer program) and / or data using machine-readable media (also called computer-readable media), such as machine -readable storage media (e.g., magnetic disks, optical disks, solid state drives, read only memory (ROM), flash memory devices, phase change memory) and machine-readable transmission media (also called a carrier) (e.g., electrical, optical, radio, acoustical, or other form of propagated signals - such as carrier waves, infrared signals). Thus, an electronic device (e.g., a computer) includes hardware and software, such as a set of one or more processors (e.g., of which a processor is a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), other electronic circuitry, or a combination of one or more of the preceding) coupled to one or more machine-readable storage media to store code for execution on the set of processors and / or to store data. For instance, an electronic device may include non-volatile memory containing the code since the non-volatile memory can persist code / data even when the electronic device is turned off (when power is removed). When the electronic device is turned on, that part of the code that is to be executed by the processor(s) of the electronic device is typically copied from the slower non-volatile memory into volatile memory (e.g., dynamic random-access memory (DRAM), static random-access memory (SRAM)) of the electronic device. Typical electronic devices also include a set of one or more physical network interface(s) (NI(s)) to establish network connections (to transmit and / or receive code and / or data using propagating signals) with other electronic devices. For example, the set of physical NIs (or the set of physical NI(s) in combination with the set of processors executing code) may perform any formatting, coding, or translating to allow the electronic device to send and receive data whether over a wired and / or a wireless connection. In some embodiments, a physical NI may comprise radio circuitry capable of (1) receiving data from other electronic devices over a wireless connection and / or (2) sending data out toother devices through a wireless connection. This radio circuitry may include transmitter(s), receiver(s), and / or transceiver(s) suitable for radio frequency communication. The radio circuitry may convert digital data into a radio signal having the proper parameters (e.g., frequency, timing, channel, bandwidth, and so forth). The radio signal may then be transmitted through antennas to the appropriate recipient(s). In some embodiments, the set of physical NI(s) may comprise network interface controller(s) (NICs), also known as a network interface card, network adapter, or local area network (LAN) adapter. The NIC(s) may facilitate connecting the electronic device to other electronic devices allowing them to communicate with wire through plugging in a cable to a physical port connected to an NIC. One or more parts of an embodiment of the invention may be implemented using different combinations of software, firmware, and / or hardware.

[0114] The terms “module,” “logic,” and “unit” used in the present application, may refer to a circuit for performing the function specified. In some embodiments, the specified function may be performed by a circuit in combination with software such as by software executed by a general-purpose processor.

[0115] Any appropriate steps, methods, features, functions, or benefits disclosed herein may be performed through one or more functional units or modules of one or more virtual apparatuses. Each virtual apparatus may comprise a number of these functional units. These functional units may be implemented via processing circuitry, which may include one or more microprocessors or microcontrollers, as well as other digital hardware, which may include digital signal processors (DSPs), special-purpose digital logic, and the like. The processing circuitry may be configured to execute program code stored in memory, which may include one or several types of memory such as read-only memory (ROM), random-access memory (RAM), cache memory, flash memory devices, optical storage devices, etc. Program code stored in memory includes program instructions for executing one or more telecommunications and / or data communications protocols as well as instructions for carrying out one or more of the techniques described herein. In some implementations, the processing circuitry may be used to cause the respective functional unit to perform corresponding functions according to one or more embodiments of the present disclosure.

[0116] The term unit may have conventional meaning in the field of electronics, electrical devices, and / or electronic devices and may include, for example, electrical and / or electronic circuitry, devices, modules, processors, memories, logic solid state and / or discrete devices, computer programs or instructions for carrying out respective tasks, procedures,computations, outputs, and / or displaying functions, and so on, as such as those that are described herein.

Claims

CLAIMS:

1. A method comprising:determining (402) that a Label Distribution Protocol (LDP) session for receiving packets of a traffic flow by a first router from a second router has failed based on a detection by the second router, the packets of the traffic flow having been forwarded to reach the first router as an egress router of the traffic flow; and transmitting (404), by the second router, a label withdraw message to a third router from which the second router receives packets of the traffic flow, the label withdraw message including a first type-length-value (TLV) that identifies the first router and that is to cause the third router to withdraw labels for a plurality of Forwarding Equivalence Classes (FECs) that map to the first router.

2. The method of claim 1, wherein the first TLV is a sub-TLV that indicates an identifier of the first router to identify the first router.

3. The method of claim 2, wherein the sub-TLV includes a type field indicating the sub-TLV is for identifying the first router, a length field indicating length of the sub-TLV, and a value field indicating the identifier of the first router.

4. The method of claim 3, wherein the type field is two bytes long, the length field is four bytes long, and the value field is four bytes long.

5. The method of any of claims 1 to 4, wherein the first TLV is generated based on a second TLV that is included in a first label mapping message transmitted by the first router and propagated from the first router to remaining routers through which the traffic flow is forwarded.

6. The method of claim 5, wherein propagating the second TLV comprises: upon a router receiving the first label mapping message, the router stores an identifier of the first router, one or more FECs mapped to the first router, and corresponding labels.

7. The method of claim 6, wherein propagating the second TLV further comprises transmitting a second label mapping message from the router to another router that is further away to the first router in a label switched path including the LDP session for the traffic flow, the second label mapping message including the second TLV.

8. The method of any of claims 1 to 7, wherein the plurality of FECs are represented by a single wildcard FEC.

9. The method of any of claims 1 to 8, wherein withdrawing the labels for the plurality of FECs comprises removing the plurality of FECs in a label forwarding table (LFIB) of the third router.

10. The method of any of claims 1 to 8, wherein failure of the LDP session is due to degradation of links or routers involved in the LDP session.

11. A network device (502, 504), comprising:a processor (512, 542) and machine-readable storage medium (518, 548) that provides instructions that, when executed by the processor (512, 542), are capable of causing the processor (512, 542) to perform:determining (402) that a Label Distribution Protocol (LDP) session for receiving packets of a traffic flow by a first other network device from the network device has failed, the packets of the traffic flow having been forwarded to reach the first other network device as an egress other network device of the traffic flow; andtransmitting (404) a label withdraw message to a second other network device from which the network device receives packets of the traffic flow, the label withdraw message including a first type-length-value (TLV) that identifies the first other network device and that is to cause the second other network device to withdraw labels for a plurality of Forwarding Equivalence Classes (FECs) that map to the first other network device.

12. The network device of claim 11, wherein the first TLV is a sub-TLV that indicates an identifier of the first other network device to identify the first other network device.

13. The network device of claim 12, wherein the sub-TLV includes a type field indicating the sub-TLV is for identifying the first other network device, a length field indicating length of the sub-TLV, and a value field indicating the identifier of the first other network device.

14. The network device of claim 13, wherein the type field is two bytes long, the length field is four bytes long, and the value field is four bytes long.

15. The network device of any of claims 11 to 14, wherein the first TLV is generated based on a second TLV that is included in a first label mapping message transmitted by the first other network device and propagated from the first other network device to remaining other network devices through which the traffic flow is forwarded.

16. The network device of claim 15, wherein propagating the second TLV comprises: upon a other network device receiving the first label mapping message, the other network device stores an identifier of the first other network device, one or more FECs mapped to the first other network device, and corresponding labels.

17. The network device of claim 16, wherein propagating the second TLV further comprises transmitting a second label mapping message from the other network device to another other network device that is further away to the first other network device, the second label mapping message including the second TLV.

18. The network device of any of claims 11 to 17, wherein the plurality of FECs are represented by a single wildcard FEC.

19. The network device of any of claims 11 to 17, wherein withdrawing the labels for the plurality of FECs comprises removing the plurality of FECs in a label forwarding table (LFIB) of the second other network device.

20. The network device of any of claims 11 to 19, wherein failure of the LDP session is due to degradation of link or routers involved in the LDP session.

21. A machine-readable storage medium (518, 548) that provides instructions that, when executed by a processor (512, 542), are capable of causing the processor (512, 542) to perform methods 1 to 10.

Images