Traffic blackhole avoidance during ethernet virtual private network (EVPN) mac mobility

Abstract

A method performed by a first provider edge (PE) in an ethernet virtual private network (EVPN) is disclosed to handle media access control (MAC) mobility. The method includes receiving a first route advertisement message from a second PE, wherein the first route advertisement message includes a MAC address associated with a host, a first ethernet segment identifier (ESI), and a sequence number, receiving a second route advertisement message from a third PE, wherein the second route advertisement message includes the MAC address, a second ESI, and the same sequence number included in the first route advertisement message, and responsive to a determination that the route advertisement messages include the same MAC address and the same sequence number but include different ESIs, invalidating routes associated with the MAC address having the same sequence number and removing an entry for the MAC address from a MAC table.

Description

TRAFFIC BLACKHOLE AVOIDANCE DURING ETHERNET VIRTUAL PRIVATE NETWORK (EVPN) MAC MOBILITYTECHNICAL FIELD

[0001] Embodiments of the invention relate to the field of computer networks, and more specifically to handling MAC mobility in an ethemet virtual private network (EVPN).BACKGROUND

[0002] Ethemet virtual private network (EVPN) is a technology for carrying layer 2 ethernet / IP (internet protocol) traffic as a virtual private network using wide area network protocols. EVPN technologies include ethernet over multi-protocol label switching (MPLS), ethernet over virtual extensible local area network (VXLAN), and ethernet over segment routing over IP version 6 (IPv6) dataplane (SRv6).

[0003] An EVPN instance may include customer edges (CEs) that are connected to provider edges (PEs) that form the edge of a network infrastructure. PEs may be connected to other PEs over a multiprotocol label switching (MPLS) infrastructure, which provides the benefits of MPLS technology, such as fast reroute and resiliency. Alternatively, PEs may be connected to other PEs over an internet protocol (IP) infrastructure, in which case generic routing encapsulation (GRE) tunneling, SRv6, or other IP tunneling can be used between the PEs. A CE may be a host, a router, or a switch. The PEs may provide virtual layer 2 bridged connectivity between the CEs. An EVPN may include multiple EVPN instances.

[0004] When a customer site is connected to one or more PEs via a set of ethemet links, then this set of ethernet links constitutes an "ethemet segment." Each ethernet segment may be identified by an identifier called an ethernet segment identifier (ESI).

[0005] It is possible for a given host or end station (as defined by its MAC address) to move from one ethemet segment to another- this may be referred to as “MAC Mobility” or “MAC move.”

[0006] Existing EVPN solutions provide MAC mobility procedures to allow PEs in an EVPN instance to correctly determine how to reach a host media access control (MAC) address (e.g., as described in request for comment (RFC) 7432). In a situation where there are multiple moves of a given MAC address, there may be multiple withdrawals and re-advertisements. In order to ensure that all PEs in the EVPN instance receive all of these correctly through the intervening BGP infrastructure, existing MAC mobility procedures add a sequence number into the MAC mobility extended community attribute.

[0007] According to existing MAC mobility procedures, a PE receiving a MAC / IP advertisement route for a MAC address with a different ethernet segment identifier and a higher sequence number than that which it had previously advertised withdraws its MAC / IP advertisement route. If two (or more) PEs advertise the same MAC address with the same sequence number but different ethernet segment identifiers, a PE that receives these routes selects the route advertised by the PE with the lowest IP address as the best route. If the PE is the originator of the MAC route and it receives the same MAC address with the same sequence number that it generated, it will compare its own IP address with the IP address of the remote PE and will select the lowest IP. If its own route is not the best one, it will withdraw the route. The expectation is that the PE advertising the MAC / IP route with the highest sequence number is the correct target for the host MAC address. However, the use of existing MAC mobility procedures can lead to a traffic blackholing scenario where a PE sends traffic destined for the MAC address to the incorrect PE.SUMMARY

[0008] A method performed by a network device functioning as a first provider edge (PE) in an ethernet virtual private network (EVPN) is disclosed to handle media access control (MAC) mobility in the EVPN. The method includes receiving a first route advertisement message from a second PE in the EVPN, wherein the first route advertisement message includes a MAC address associated with a host, a first ethernet segment identifier identifying a first ethernet segment, and a sequence number, receiving a second route advertisement message from a third PE in the EVPN, wherein the second route advertisement message includes the MAC address, a second ethernet segment identifier identifying a second ethernet segment, and the same sequence number included in the first route advertisement message, and responsive to a determination that the first route advertisement message and the second route advertisement message include the same MAC address and the same sequence number but include different ethernet segment identifiers, invalidating routes associated with the MAC address having the same sequence number and removing an entry for the MAC address from a MAC table.

[0009] A method performed by a network device functioning as a first provider edge (PE) in an ethernet virtual private network (EVPN) is disclosed to handle media access control (MAC) mobility in the EVPN. The method includes sending a first route advertisement message in the EVPN, wherein the first route advertisement message includes a MAC address associated with a host, a first ethernet segment identifier identifying a first ethernet segment, and a sequence number, receiving a second route advertisement message from a second PE in the EVPN, wherein the second route advertisement message includes the MAC address, a second ethernet segmentidentifier identifying a second ethernet segment, and the same sequence number included in the first route advertisement message, and responsive to a determination that the second route advertisement message includes the same MAC address and sequence number included in the first route advertisement message but a different ethernet segment identifier, sending a withdrawal message for the MAC address in the EVPN, removing an entry for the MAC address from a MAC table, and invalidating routes associated with the MAC address having the same sequence number.

[0010] A non-transitory machine -readable storage medium is disclosed that provides instructions that, if executed by a processor of a network device implementing a first provider edge (PE) in an ethernet virtual private network (EVPN), will cause the first PE to carry out operations for handling media access control (MAC) mobility in the EVPN. The operations include receiving a first route advertisement message from a second PE in the EVPN, wherein the first route advertisement message includes a MAC address associated with a host, a first ethernet segment identifier identifying a first ethernet segment, and a sequence number, receiving a second route advertisement message from a third PE in the EVPN, wherein the second route advertisement message includes the MAC address, a second ethernet segment identifier identifying a second ethernet segment, and the same sequence number included in the first route advertisement message, and responsive to a determination that the first route advertisement message and the second route advertisement message include the same MAC address and the same sequence number but include different ethernet segment identifiers, invalidating routes associated with the MAC address having the same sequence number and removing an entry for the MAC address from a MAC table..

[0011] A non-transitory machine -readable storage medium is disclosed that provides instructions that, if executed by a processor of a network device implementing a first provider edge (PE) in an ethernet virtual private network (EVPN), will cause the first PE to carry out operations for handling media access control (MAC) mobility in the EVPN. The operations include sending a first route advertisement message in the EVPN, wherein the first route advertisement message includes a MAC address associated with a host, a first ethernet segment identifier identifying a first ethernet segment, and a sequence number, receiving a second route advertisement message from a second PE in the EVPN, wherein the second route advertisement message includes the MAC address, a second ethernet segment identifier identifying a second ethernet segment, and the same sequence number included in the first route advertisement message, and responsive to a determination that the second route advertisement message includes the same MAC address and sequence number included in the first route advertisement message but a different ethernet segment identifier, sending a withdrawal message for the MAC address inthe EVPN, removing an entry for the MAC address from a MAC table, and invalidating routes associated with the MAC address having the same sequence number.

[0012] A network device is disclosed to implement a first provider edge (PE) in an ethemet virtual private network (EVPN) that is configured to handle media access control (MAC) mobility in the EVPN. The network device includes a set of one or more processors and a non-transitory machine-readable storage medium that provides instructions that, if executed by the set of one or more processors will cause the first PE to carry out operations including receiving a first route advertisement message from a second PE in the EVPN, wherein the first route advertisement message includes a MAC address associated with a host, a first ethernet segment identifier identifying a first ethernet segment, and a sequence number, receiving a second route advertisement message from a third PE in the EVPN, wherein the second route advertisement message includes the MAC address, a second ethernet segment identifier identifying a second ethernet segment, and the same sequence number included in the first route advertisement message, and responsive to a determination that the first route advertisement message and the second route advertisement message include the same MAC address and the same sequence number but include different ethernet segment identifiers, invalidating routes associated with the MAC address having the same sequence number and removing an entry for the MAC address from a MAC table.

[0013] A network device is disclosed to implement a first provider edge (PE) in an ethernet virtual private network (EVPN) that is configured to handle media access control (MAC) mobility in the EVPN. The network device includes a set of one or more processors and a non-transitory machine-readable storage medium that provides instructions that, if executed by the set of one or more processors will cause the first PE to carry out operations including sending a first route advertisement message in the EVPN, wherein the first route advertisement message includes a MAC address associated with a host, a first ethemet segment identifier identifying a first ethernet segment, and a sequence number, receiving a second route advertisement message from a second PE in the EVPN, wherein the second route advertisement message includes the MAC address, a second ethernet segment identifier identifying a second ethemet segment, and the same sequence number included in the first route advertisement message, and responsive to a determination that the second route advertisement message includes the same MAC address and sequence number included in the first route advertisement message but a different ethernet segment identifier, sending a withdrawal message for the MAC address in the EVPN, removing an entry for the MAC address from a MAC table, and invalidating routes associated with the MAC address having the same sequence number.

[0014] Embodiments disclosed herein may be used to avoid the traffic blackholing that occurs with existing MAC mobility procedures.BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The invention may best be understood by referring to the following description and accompanying drawings that are used to illustrate embodiments of the invention. In the drawings:

[0016] Figure 1 is a diagram showing a traffic blackholing scenario that can occur in an ethernet virtual private network (EVPN) when using existing EVPN media access control (MAC) mobility procedures.

[0017] Figure 2 is a diagram showing how a remote provider edge (PE) handles MAC mobility, according to some embodiments.

[0018] Figure 3 is a diagram showing how originating PEs handle MAC mobility, according to some embodiments.

[0019] Figure 4 is a diagram showing how PEs learn the correct route for a MAC address, according to some embodiments.

[0020] Figure 5 is a diagram showing components of a PE and their interactions to handle MAC mobility, according to some embodiments.

[0021] Figure 6 is a diagram showing PE interactions to handle MAC mobility, according to some embodiments.

[0022] Figure 7 is a flow diagram of a method for handling MAC mobility, according to some embodiments.

[0023] Figure 8 is a flow diagram of a method for handling MAC mobility, according to some embodiments.

[0024] Figure 9A illustrates connectivity between network devices (NDs) within an exemplary network, as well as three exemplary implementations of the NDs, according to some embodiments.

[0025] Figure 9B illustrates an exemplary way to implement a special-purpose network device according to some embodiments.DETAILED DESCRIPTION

[0026] The following description describes methods and apparatus for handling media access control (MAC) mobility in an ethernet virtual private network (EVPN). In the following description, numerous specific details such as logic implementations, opcodes, means to specify operands, resource partitioning / sharing / duplication implementations, types and interrelationships of system components, and logic partitioning / integration choices are set forth in order to providea more thorough understanding of the present invention. It will be appreciated, however, by one skilled in the art that the invention may be practiced without such specific details. In other instances, control structures, gate level circuits and full software instruction sequences have not been shown in detail in order not to obscure the invention. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate functionality without undue experimentation .

[0027] References in the specification to “one embodiment,” “an embodiment,” “an example embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

[0028] Bracketed text and blocks with dashed borders (e.g., large dashes, small dashes, dotdash, and dots) may be used herein to illustrate optional operations that add additional features to embodiments of the invention. However, such notation should not be taken to mean that these are the only options or optional operations, and / or that blocks with solid borders are not optional in certain embodiments of the invention.

[0029] In the following description and claims, the terms “coupled” and “connected,” along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. “Coupled” is used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, co-operate or interact with each other. “Connected” is used to indicate the establishment of communication between two or more elements that are coupled with each other.

[0030] An electronic device stores and transmits (internally and / or with other electronic devices over a network) code (which is composed of software instructions and which is sometimes referred to as computer program code or a computer program) and / or data using machine-readable media (also called computer-readable media), such as machine-readable storage media (e.g., magnetic disks, optical disks, solid state drives, read only memory (ROM), flash memory devices, phase change memory) and machine -readable transmission media (also called a carrier) (e.g., electrical, optical, radio, acoustical or other form of propagated signals - such as carrier waves, infrared signals). Thus, an electronic device (e.g., a computer) includes hardware and software, such as a set of one or more processors (e.g., wherein a processor is a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application specific integratedcircuit, field programmable gate array, other electronic circuitry, a combination of one or more of the preceding) coupled to one or more machine-readable storage media to store code for execution on the set of processors and / or to store data. For instance, an electronic device may include nonvolatile memory containing the code since the non-volatile memory can persist code / data even when the electronic device is turned off (when power is removed), and while the electronic device is turned on that part of the code that is to be executed by the processor(s) of that electronic device is typically copied from the slower non-volatile memory into volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM)) of that electronic device. Typical electronic devices also include a set of one or more physical network interface(s) (NI(s)) to establish network connections (to transmit and / or receive code and / or data using propagating signals) with other electronic devices. For example, the set of physical NIs (or the set of physical NI(s) in combination with the set of processors executing code) may perform any formatting, coding, or translating to allow the electronic device to send and receive data whether over a wired and / or a wireless connection. In some embodiments, a physical NI may comprise radio circuitry capable of receiving data from other electronic devices over a wireless connection and / or sending data out to other devices via a wireless connection. This radio circuitry may include transmitter(s), receiver(s), and / or transceiver(s) suitable for radiofrequency communication. The radio circuitry may convert digital data into a radio signal having the appropriate parameters (e.g., frequency, timing, channel, bandwidth, etc.). The radio signal may then be transmitted via antennas to the appropriate recipient(s). In some embodiments, the set of physical NI(s) may comprise network interface controller(s) (NICs), also known as a network interface card, network adapter, or local area network (LAN) adapter. The NIC(s) may facilitate in connecting the electronic device to other electronic devices allowing them to communicate via wire through plugging in a cable to a physical port connected to a NIC. One or more parts of an embodiment of the invention may be implemented using different combinations of software, firmware, and / or hardware.

[0031] A network device (ND) is an electronic device that communicatively interconnects other electronic devices on the network (e.g., other network devices, end-user devices). Some network devices are “multiple services network devices” that provide support for multiple networking functions (e.g., routing, bridging, switching, Layer 2 aggregation, session border control, Quality of Service, and / or subscriber management), and / or provide support for multiple application services (e.g., data, voice, and video).

[0032] As mentioned above, the use of existing MAC mobility procedures can lead to a traffic blackholing scenario where the PE sends traffic destined for the MAC address to the incorrect PE. An example of a traffic blackholing scenario is shown in Figure 1 and described in relation thereto.

[0033] Figure 1 is a diagram showing a traffic blackholing scenario that can occur in an EVPN when using existing EVPN MAC mobility procedures.

[0034] As shown in the diagram, the EVPN includes a host 110, CEs 120 (CE1 120A, CE2 120B, and CE3 120C), PEs 130 (PEI 130A, PE2 BOB, and PE3 130C), and a network infrastructure 150. PEs 130 may be communicatively coupled to other PEs 130 over a network infrastructure 150. In an embodiment, the network infrastructure 150 is a multiprotocol label switching (MPLS) infrastructure, which provides the benefits of MPLS technology, such as fast reroute and resiliency. However, it should be appreciated that the network infrastructure 150 may be implemented using other types of network virtualization overlay (NVO) techniques such as virtual extensible local area network (VXLAN), network virtualization using generic routing encapsulation (NVGRE), generic network virtualization encapsulation (GENEVE), or segment routing over IP version 6 (IPv6) dataplane (SRv6). In the example shown in the diagram, PEI 130A has an IP address of 1.1.1.1, PE2 130B has an IP address of 2.2.2.2, and PE3 130C has an IP address of 3.3.3.3.

[0035] A CE 120 may be, for example, a host, a router, or a switch. In the example environment shown in the diagram, CE1 BOA is attached to PEI BOA via ethernet segment ESH, CE2 BOB is attached to PE2 BOB via ethernet segment ESI2, and CE3 BOB is attached to PE3 130C via ethernet segment ESI3.

[0036] While the diagram shows the EVPN as having a particular arrangement of components, it should be appreciated that the EVPN can have a different arrangement than shown in the diagram (e.g., the EVPN system may include additional PEs 130, additional CEs 120, and / or other network components). Thus, the particular arrangement shown in the diagram should be regarded as illustrative and not regarded as limiting.

[0037] In the example shown in the diagram, at time tl, the host 110 is connected to CE1 BOA. The host 110 may be a network device that can move locations. In this example, the host 110 has a MAC address of ML PEI BOA may leam about the host’s 110 MAC address Ml via data plane learning and advertise a MAC / IP route for Ml in the EVPN (advertise the MAC / IP route to other PEs 130 via the network infrastructure 150) with ESI1 and sequence number 0 (zero). In an embodiment, a PE 130 advertises a MAC / IP route using a MP-BGP (multi-protocol border gateway protocol) message that includes a “MAC / IP Advertisement route” (also referred to as an EVPN Type 2 route). In an embodiment, a sequence number is indicated in a MAC mobility extended community included in the MP-BGP message. In response to receiving the advertisement of the MAC / IP route for Ml, PE2 BOB and PE3 130C may add an entry for Ml to their respective MAC tables with PEI BOA as the next hop. A MAC table may be a table inhardware that includes entries for MAC addresses that indicate how to forward traffic destined for those MAC addresses.

[0038] In the example shown in the diagram, it is assumed that at time t2, the host 110 moves locations such that the host 110 is connected to CE2 120B instead of CE1 120A. PE2 130B may learn about the host’s 110 MAC address Ml via data plane learning (e.g., from CE2 120B) and advertise a MAC / IP route for Ml in the EVPN with ESI2 and sequence number 1 (one). PE2 130B advertises the MAC / IP route with sequence number 1 (one) because it previously received a MAC / IP route advertisement for Ml with sequence number 0 (zero). The PEs 130 that receive the MAC / IP route advertisement are expected to add an entry for Ml to their respective MAC tables with PE2 130B as next hop.

[0039] As shown in the diagram, it is assumed that at time t3, the host 110 moves locations such that the host 110 is connected to CE3 120C instead of CE2 120B. If PE3 130C received PE2’s 130B MAC / IP route advertisement by this time, then PE3 130C may leam about the host’s 110 MAC address Ml via data plane learning (e.g., from CE3 120C) and advertise a MAC / IP route for Ml in the EVPN with ESI3 and sequence number 2 (two) (since it received a MAC / IP route advertisement for Ml from PE2 130B with sequence number 1 (one)). In this scenario, MAC mobility works as expected and there is no traffic blackholing.

[0040] However, if PE3 130C learns about the host’s 110 MAC address Ml before it receives and processes PE2’s 130B MAC / IP route advertisement (which has sequence number 1 (one)), then the MAC / IP route for Ml stored at PE3 130C still has sequence number 0 (zero) (as learned from PEI 130A). This may happen due to PE2’s 130B MAC / IP route advertisement being delayed or missing. In this situation, PE3 130B may advertise a MAC / IP route for Ml in the EVPN with ESI3 and sequence number 1 (one). After some time, PE3 130C may eventually receive the MAC / IP route advertised by PE2 130B with sequence number 1 (one). Also, PE2 130B may receive the MAC / IP route advertised by PE3 130C with sequence number 1 (one). In this situation, there are two MAC / IP route advertisements with the same sequence number in the EVPN domain.

[0041] According to existing EVPN MAC mobility procedures (e.g., as described in request for comment (RFC) 7432), if two (or more) PEs 130 advertise the same MAC address with the same sequence number but different ethernet segment identifiers, a PE 130 that receives these routes selects the route advertised by the PE 130 with the lowest IP address as the best route. If the PE 130 is the originator of the MAC route and it receives the same MAC address with the same sequence number that it generated, it will compare its own IP address with the IP address of the remote PE 130 and will select the lowest IP address. If its own route is not the best one, it will withdraw the route.

[0042] If the existing EVPN MAC mobility procedures are followed, at time t4, PEI 130A may have learned two routes for Ml with the same sequence number: (1) a route with PE2 130B as the next hop; and (2) a route with PE3 130C as the next hop (the sequence number is shown in parenthesis in the diagram). However, PEI 130A may decide to use the first route (the route with PE2 130B as the next hop) (as denoted by the symbol in the diagram) because PE2 130B has a lower IP address compared to PE3 130C (PE2 130B has IP address of 2.2.2.2 and PE3 130C has IP address of 3.3.3.3). Thus, at time t4, PEI 130A may send traffic destined for Ml to PE2 130B. PE2 130B may then attempt to send the traffic to CE2 120B. However, the host 110 is no longer connected to CE2 120B and thus CE2 120B is not able to send the traffic to the host 110. This traffic “blackholing” may occur until PE3 130C learns about the host’s 110 MAC address Ml again (and advertises a MAC / IP route with a higher sequence number) or the entry for Ml in PE2’s 130B MAC table ages out. Typically, the time it takes for an entry in the MAC table to age out is in order of a few minutes. Thus, there may be traffic disruption during this time.

[0043] Embodiments are disclosed herein that address the traffic blackholing problem. According to embodiments, when a remote PE 130 (e.g., PEI 130A in the example) receives two MAC / IP route advertisements for a MAC address with different ESIs and the same sequence number, it discards / invalidates the MAC / IP routes (e.g., does not consider the MAC / IP routes for Ml advertised by PE2 130B and PE3 130C when determining best path in BGP) since the source of the MAC address (e.g., Ml) cannot be decided correctly. Instead, the remote PE sends traffic destined for the MAC address as broadcast, unknown unicast, or multicast (BUM) traffic (e.g., using a BUM label to flood the traffic to all PEs 130). The correct PE 130 (e.g., PE 130C in this example) may then send the traffic to the correct CE 120 (e.g., CE3 120C in this example). This ensures that the traffic is not disrupted.

[0044] According to embodiments, when an originating PE 130 (e.g., PE2 130B and PE3 130B in this example) receives a MAC / IP route advertisement for a MAC address with the same sequence number but different ESI compared to the MAC / IP route advertised by the originating PE 130, the originating PE 130 may withdraw its MAC / IP route and also “flush” (e.g., forcefully age out) the entry for the MAC address in its MAC table so that the originating PE 130 floods traffic destined for the MAC address to its attached CEs 120 when it receives traffic from remote PE using BUM label or from local attached circuits.

[0045] According to embodiments, once the PE 130 to which the host is connected learns about the MAC address again, that PE 130 shall advertise a MAC / IP route with a higher sequence number. The PEs 130 that receive the advertisement shall add an entry for the MAC address to their respective MAC tables with the advertising PE as the next hop. This way, there will not beany traffic blackholing during MAC mobility. This improves the robustness of the EVPN, especially during quick MAC moves.

[0046] An embodiment is a method performed by a (remote) PE in an EVPN to handle MAC mobility in the EVPN. The method includes receiving a first route advertisement message from a second PE in the EVPN, wherein the first route advertisement message includes a MAC address associated with a host, a first ethemet segment identifier identifying a first ethernet segment, and a sequence number, receiving a second route advertisement message from a third PE in the EVPN, wherein the second route advertisement message includes the MAC address, a second ethernet segment identifier identifying a second ethernet segment, and the same sequence number included in the first route advertisement message, and responsive to a determination that the first route advertisement message and the second route advertisement message include the same MAC address and the same sequence number but include different ethernet segment identifiers, invalidating routes associated with the MAC address having the same sequence number and removing an entry for the MAC address from a MAC table.

[0047] An embodiment is a method performed by an (originating) PE in an EVPN to handle MAC mobility in the EVPN. The method includes sending a first route advertisement message in the EVPN, wherein the first route advertisement message includes a MAC address associated with a host, a first ethernet segment identifier identifying a first ethemet segment, and a sequence number, receiving a second route advertisement message from a second PE in the EVPN, wherein the second route advertisement message includes the MAC address, a second ethernet segment identifier identifying a second ethernet segment, and the same sequence number included in the first route advertisement message, and responsive to a determination that the second route advertisement message includes the same MAC address and sequence number included in the first route advertisement message but a different ethernet segment identifier, sending a withdrawal message for the MAC address in the EVPN, removing an entry for the MAC address from a MAC table, and invalidating routes associated with the MAC address having the same sequence number.

[0048] Embodiments disclosed herein provide one or more advantages over the existing EVPN MAC mobility procedures. As an example, embodiments allow the remote PE to continue traffic delivery to the host without disruption. Also, as an example, embodiments prevent the originating PEs from using stale MAC table entries for sending traffic, which avoids traffic loss. Other advantages will be apparent to those skilled in the relevant art in view of the present disclosure.

[0049] Figure 2 is a diagram showing how a remote PE handles MAC mobility, according to some embodiments.

[0050] The example shown in the diagram is a continuation of the example scenario shown in Figure 1 after time tl . As shown in the diagram, at time t2, the host 110 is connected to CE2 120B(which is attached to PE 130B). PE2 130B learns about the host’s 110 MAC address Ml and advertises a MAC / IP route for Ml in the EVPN with ESI2 and sequence number 1 (one). At time t3, the host 110 moves locations such that the host 110 is connected to CE3 120C (which is attached to PE3 130C) instead of CE2 120B (which is attached to PE2 130B). In this example, it is assumed that PE3 130C learns about the host’s 110 MAC address Ml (via data plane learning) before it receives and processes PE2’s 130B MAC / IP route advertisement for Ml. Thus, PE3 130C advertises a MAC / IP route for Ml in the EVPN with ESI3 and sequence number 1 (one).

[0051] At time t4, PEI 130A determines that it received two MAC / IP route advertisements for the same MAC address (Ml) with different ESIs (ESI2 and ESI3) but having the same sequence number (sequence number 1 (one)). PEI 130A is considered a remote PE in this situation (since it did not originate the routes), while PE2 130B and PE3 130C are considered originating PEs in this situation (since they originated the routes). In response to determining that it received two MAC / IP route advertisements for the same MAC address (Ml) with different ESIs (ESI2 and ESI3) but having the same sequence number, PEI 130A invalidates both of the MAC / IP routes for Ml (as shown with strikethrough in the diagram). Assuming PEI 130A has not received a MAC / IP route advertisement for Ml with a higher sequence number, PEI 130A removes the entry for Ml from its MAC table. As a result, PEI 130A sends traffic destined for Ml as BUM traffic (e.g., using a BUM label).

[0052] Figure 3 is a diagram showing how originating PEs handle MAC mobility, according to some embodiments.

[0053] The example shown in the diagram is a continuation of the example scenario shown in Figure 1 after time tl . As shown in the diagram, at time t2, the host 110 is connected to CE2 120B (which is attached to PE 130B). PE2 130B learns about the host’s 110 MAC address Ml and advertises a MAC / IP route for Ml with ESI2 and sequence number 1 (one). At time t3, the host 110 moves locations such that the host 110 is connected to CE3 120C (which is attached to PE3 130C) instead of CE2 120B (which is attached to PE2 130B). In this example, it is assumed that PE3 130C leams about the host’s 110 MAC address Ml (via data plane learning) before it receives PE2’s 130B MAC / IP route advertisement for Ml. Thus, PE3 130C advertises a MAC / IP route for Ml with ESI3 and sequence number 1 (one).

[0054] At time t4, PE2 130B receives PE3’s 130C MAC / IP route advertisement and determines that it had originated a MAC / IP route for the same MAC address (Ml) with a different ESI (ESI2 instead of ESI3) but the same sequence number (sequence number 1 (one)). In response to this determination, PE2 130B withdraws its MAC / IP route for Ml, invalidates MAC / IP routes for Ml having the same sequence number (as shown with strikethrough in the diagram), and removes the entry for Ml from its MAC table. PE2 130B may withdraw the MAC / IP route for Ml by sendinga withdraw message for Ml in the EVPN (e.g., to PEI 130A and PE3 130C). In an embodiment, the withdraw message is a MP-BGP message that includes multiprotocol unreachable network layer reachability information (MP_UNREACH_NLRI).

[0055] At time t5, PE3 130C receives PE2’s 130B (delayed) MAC / IP route advertisement and determines that it had originated a MAC / IP route for the same MAC address (Ml) with a different ESI (ESI3 instead of ESI2) but the same sequence number (sequence number 1 (one)). In response to this determination, PE3 130C withdraws its MAC / IP route for Ml, invalidates MAC / IP routes for Ml having the same sequence number (as shown with strikethrough in the diagram), and removes the entry for Ml from its MAC table. PE3 130C may withdraw the MAC / IP route by sending a withdraw message for Ml in the EVPN (e.g., to PEI 130A and PE2 130B).

[0056] As a result, when PE2 130B and PE3 130C receive traffic destined for Ml from a remote PE 130, they may flood the traffic to their attached CEs 120 (due to the absence of an entry for Ml in their MAC tables). Also, when PE2 130B and PE3 130C receive traffic destined for Ml from a locally attached CE 120, they may flood the traffic to other PEs 130 (e.g., via the network 150 using a BUM label).

[0057] Figure 4 is a diagram showing how PEs learn the correct route for a MAC address, according to some embodiments.

[0058] The example shown in the diagram is a continuation of the example scenario shown in Figure 3 after time t5. As shown in the diagram, at time t6, the host 110 is connected to CE3 120C (which is attached to PE3 130C). PE3 130C learns about the host’s 110 MAC address Ml (e.g., via data plane learning), creates an entry for Ml in its MAC table, and advertises a MAC / IP route for Ml with ESI3 and sequence number 2 (two).

[0059] At time t7, in response to receiving the MAC / IP route advertisement for Ml, PEI 130A and PE2 130B add an entry for Ml to their respective (hardware) MAC tables with PE3 130C as the next hop. As a result, PEI 130A and PE2 130B may send traffic destined for Ml to PE3 130C (which is the correct PE 130). Also, PE3 130C may send traffic destined for Ml to CE3 120C via ESI3.

[0060] Figure 5 is a diagram showing components of a PE and their interactions to handle MAC mobility, according to some embodiments.

[0061] As shown in the diagram, the PE 130 includes a BGP component 510, a bridge component 520, and a hardware component 530. The BGP component 510 may be configured to perform operations in accordance with MP-BGP or similar protocol. The BGP component 510 may maintain information regarding routes 515 (e.g., information regarding routes learned via MAC / IP route advertisements or data plane learning). The route information 515 may include MAC addresses, IP addresses, label information (e.g., MPLS labels), ESIs, and / or sequencenumbers. The bridge component 520 may maintain the MAC entries leamt from remote PE’s or due to local data plane learning. The hardware component 530 may include a MAC table 535. The MAC table 535 may include entries for MAC addresses that indicate how to forward traffic destined for those MAC addresses.

[0062] Operations L1-L3 shown in the diagram are operations for learning a local route.

[0063] At operation LI, the hardware component 530 learns about a MAC address via data plane learning and adds an entry for the MAC address in the MAC table 535. The entry may indicate how to forward traffic destined for the MAC address.

[0064] At operation L2, the hardware component 530 updates the bridge component 520 with the local learning. The bridge component 520 may then update the BGP component 510.

[0065] At operation L3, the BGP component 510 may generate a route advertisement message for the MAC / IP route. The BGP component 510 may store information regarding the MAC / IP route (as part of route information 515).

[0066] Operations R1-R5 are operations for learning a remote route.

[0067] At operation Rl, the BGP component receives an advertisement of a MAC / IP route.

[0068] At operation R2, the BGP component determines that the advertised MAC / IP route has a different ESI but the same MAC address and sequence number as another previously-learned MAC / IP route (e.g., that was learned based on local learning or remote learning).

[0069] At operation R3, the BGP component 510 invalidates the MAC / IP routes having the same sequence number and triggers the bridge component 520 to remove the entry for the MAC address. If the PE 130 is an originating PE, then the BGP component 510 may also generate a withdraw message for the MAC / IP route.

[0070] At operation R4, the bridge component 520 removes the entry for the MAC address from the hardware MAC table 535. For example, if the entry is a local entry (created based on local learning) then the entry may be forced to age out (flush). If the entry is a remote entry (created based on remote learning) then the entry may simply be removed.

[0071] At operation R5, the PE 130 sends traffic destined for the MAC address as BUM traffic (e.g., if the PE 130 is a remote PE) or floods the traffic to CEs (e.g., if the PE 130 is an originating PE) due to the absence of an entry for the MAC address in the MAC table 535. This may continue until the PE 130 receives a MAC / IP route advertisement for the MAC address with a higher (unique) sequence number or learns about the MAC address via data plane learning.

[0072] Figure 6 is a diagram showing PE interactions to handle MAC mobility, according to some embodiments.

[0073] As shown in the diagram, at operation 605, PEI 130A learns about a MAC address via data plane learning.

[0074] At operation 610, PEI 130A adds a local MAC / IP route (in BGP) and advertises the MAC / IP route to PE2 130B and PE3 130C. A local MAC / IP route may be a route which is learnt based on data plane learning on attached circuit / port.

[0075] In response to receiving the MAC / IP route advertisement from PEI 130A, at operation 615, PE2 130B adds a remote MAC / IP route with PEI 130A as the next hop (in BGP). Similarly, in response to receiving the MAC / IP route advertisement from PEI 130A, at operation 620, PE3 130C adds a remote MAC / IP route with PEI 130A as the next hop (in BGP). A remote MAC / IP route may be a route created in response to MAC / IP route advertisement received from other PEs 130 in the EVPN.

[0076] At operation 625, it is assumed that the host associated with the MAC address moves to PE2 130B (it is connected to a CE attached to PE2 130B) and PE2 130B learns about the MAC address via data plane learning.

[0077] At operation 630, PE2 130B adds a local MAC / IP route (in BGP) and advertises the MAC / IP route with sequence number 1 (one) to PEI 130A and PE3 130C but it is assumed in this example that the advertisement sent to PE3 130C gets delayed (but PEI 130A is able to receive the advertisement).

[0078] In response to receiving the MAC / IP route advertisement from PE2 130B, at operation 635, PEI 130A updates its MAC entry so that PE2 130B is the next hop.

[0079] At operation 640, it is assumed that the host associated with the MAC address moves to PE3 130C (it is connected to a CE attached to PE3 130C) and PE3 130C learns about the MAC address via data plane learning.

[0080] At operation 645, PE3 130C adds a local MAC / IP route (in BGP) and advertises the MAC / IP route with sequence number 1 (one) to PEI 130A and PE2 130B (PE3 130C uses sequence number 1 (one) because its last seen sequence number is sequence number 0 (zero) due to not having received PE2’s 130B MAC / IP route advertisement due to delay).

[0081] In response to receiving the MAC / IP route advertisement from PE3 130C, at operation 650, PEI 130A removes the entry for the MAC address from its MAC table due to conflict. As a result, PEI 130A sends any traffic destined for the MAC address as BUM traffic.

[0082] In response to receiving the MAC / IP route advertisement from PE3 130C, at operation 660, PE2 130B removes the entry for the MAC address from its MAC table due to conflict with PE3’s 130C MAC / IP route advertisement (same sequence number) and sends a withdraw message for the MAC address to PEI 130A and PE3 130C. Eventually, PE3 130C may receive the delayed MAC / IP route advertisement with sequence number 1 (one) from PE2 130B. At operation 665, PE3 130C removes the entry for the MAC address from its MAC table due toconflict with PE2’s 130C (delayed) MAC / IP route advertisement (same sequence number) and sends a withdraw message for the MAC address to PEI 130A and PE2 130B.

[0083] At operation 670, it is assumed that the host associated with the MAC address is still at PE3 130C and PE3 130C learns about the MAC address via data plane learning.

[0084] At operation 675, PE3 130C adds a local MAC / IP route for the MAC address (in BGP) and advertises a MAC / IP route with sequence number 2 (two) to PEI 130A and PE2 130B (PE3 130C uses sequence number 2 (two) because its last seen sequence number is sequence number 1 (one)).

[0085] In response to receiving the MAC / IP route advertisement from PE3 130C, at operation 680, PEI 130A adds a remote MAC / IP route with PE3 130C as the next hop (in BGP, assuming there is no MAC / IP route with a higher sequence number). Similarly, in response to receiving the MAC / IP route advertisement from PE3 130C, at operation 685, PE2 130B adds a remote MAC / IP route with PE3 130C as the next hop (in BGP, assuming there is no MAC / IP route with a higher sequence number). As such, PEI 130A and PE2 130B may know to send traffic destined for the MAC address to PE3 130C, which is the correct PE.

[0086] Figure 7 is a flow diagram of a method for handling MAC mobility, according to some embodiments. The method may be performed by a first PE in an EVPN.

[0087] The operations in the flow diagram will be described with reference to the exemplary embodiments of the other figures. However, it should be understood that the operations of the flow diagram can be performed by embodiments other than those discussed with reference to the other figures, and the embodiments discussed with reference to these other figures can perform operations different than those discussed with reference to the flow diagram.

[0088] Also, while the flow diagrams in the figures show a particular order of operations performed by certain embodiments, it should be understood that such order is provided by way of example and not intended to be limiting (e.g., alternative embodiments may perform the operations in a different order, combine certain operations, overlap certain operations, etc.).

[0089] At operation 710, the first PE receives a first route advertisement message from a second PE in the EVPN, wherein the first route advertisement message includes a MAC address associated with a host, a first ethemet segment identifier identifying a first ethernet segment, and a sequence number.

[0090] At operation 720, the first PE receives a second route advertisement message from a third PE in the EVPN, wherein the second route advertisement message includes the MAC address, a second ethernet segment identifier identifying a second ethemet segment, and the same sequence number included in the first advertisement message. In an embodiment, the first route advertisement message and the second route advertisement message are MP-BGP messagesincluding MAC / IP advertisement routes. In an embodiment, the same sequence number is included in MAC mobility extended communities included in the first route advertisement message and the second route advertisement message.

[0091] Responsive to a determination that the first route advertisement message and the second route advertisement message include the same MAC address and the same sequence number but include different ethernet segment identifiers, at operation 730, the first PE invalidate routes associated with the MAC address having the same sequence number (e.g., in BGP) and removes an entry for the MAC address from a MAC table (e.g., in hardware).

[0092] In an embodiment, at operation 740, the first PE receives traffic destined for the MAC address from a CE attached to the first PE.

[0093] In an embodiment, at operation 750, the first PE sends the traffic destined for the MAC address as BUM traffic in the EVPN (e.g., using a BUM label) due to an absence of the entry for the MAC address in the MAC table.

[0094] In an embodiment, at operation 760, the first PE receives a third route advertisement message from the third PE, wherein the third route message includes the MAC address, the second ethernet segment identifier, and a sequence number that is greater than the sequence number included in the first and second route advertisement messages.

[0095] In an embodiment, responsive to receiving the third route advertisement message, at operation 770, the first PE adds an entry for the MAC address in the MAC table with the third PE as a next hop.

[0096] Figure 8 is a flow diagram of a method for handling MAC mobility, according to some embodiments. The method may be performed by a first PE in an EVPN.

[0097] At operation 810, the first PE sends a first route advertisement message in the EVPN, wherein the first route advertisement message includes a MAC address associated with a host, a first ethernet segment identifier identifying a first ethernet segment, and a sequence number.

[0098] At operation 820, the first PE receives a second route advertisement message from a second PE in the EVPN, wherein the second route advertisement message includes the MAC address, a second ethernet segment identifier identifying a second ethernet segment, and the same sequence number included in the first advertisement message. In an embodiment, the first route advertisement message and the second route advertisement message are MP-BGP messages including MAC / IP advertisement routes. In an embodiment, the same sequence number is included in MAC mobility extended communities included in the first route advertisement message and the second route advertisement message.

[0099] At operation 830, responsive to a determination that the second route advertisement message includes the same MAC address and sequence number included in the first routeadvertisement message but a different ethernet segment identifier, the first PE sends a withdrawal message for the MAC address in the EVPN, removes an entry for the MAC address from a MAC table, and invalidates routes associated with the MAC address having the same sequence number. In an embodiment, the entry for the MAC address is removed from the MAC table based on forcing the entry for the MAC address to age out.

[0100] In an embodiment, at operation 840, the first PE receives traffic destined for the MAC address from another PE in the EVPN or a local attachment circuit in a same EVPN instance.

[0101] In an embodiment, at operation 850, the first PE floods the traffic destined for the MAC address to CEs attached to the first PE due to an absence of the entry for the MAC address in the MAC table.

[0102] In an embodiment, at operation 860, the first PE learns about the MAC address via data plane learning.

[0103] In an embodiment, at operation 870, responsive to the learning, the first PE creates an entry for the MAC address in the MAC table and sends a third route advertisement message in the EVPN, wherein the third route advertisement message includes the MAC address, the first ethernet segment identifier, and a sequence number that is greater than the sequence number included in the first route advertisement message and the second route advertisement message.

[0104] In an embodiment, the first PE receives a third address route advertisement message from the second PE, wherein the third route advertisement message includes the MAC address, the second ethernet segment identifier, and a sequence number that is greater than the sequence number included in the first and second route advertisement messages. In an embodiment, responsive to receiving the third address route advertisement message, the first PE adds an entry for the MAC address in the MAC table with the second PE as a next hop.

[0105] Figure 9A illustrates connectivity between network devices (NDs) within an exemplary network, as well as three exemplary implementations of the NDs, according to some embodiments of the invention. Figure 9A shows NDs 900A-H, and their connectivity by way of lines between 900A-900B, 900B-900C, 900C-900D, 900D-900E, 900E-900F, 900F-900G, and 900A-900G, as well as between 900H and each of 900A, 900C, 900D, and 900G. These NDs are physical devices, and the connectivity between these NDs can be wireless or wired (often referred to as a link). An additional line extending from NDs 900A, 900E, and 900F illustrates that these NDs act as ingress and egress points for the network (and thus, these NDs are sometimes referred to as edge NDs; while the other NDs may be called core NDs).

[0106] Two of the exemplary ND implementations in Figure 9 A are: 1) a special-purpose network device 902 that uses custom application-specific integrated-circuits (ASICs) and aspecial-purpose operating system (OS); and 2) a general purpose network device 904 that uses common off-the-shelf (COTS) processors and a standard OS.

[0107] The special-purpose network device 902 includes networking hardware 910 comprising a set of one or more processor(s) 912, forwarding resource(s) 914 (which typically include one or more ASICs and / or network processors), and physical network interfaces (NIs) 916 (through which network connections are made, such as those shown by the connectivity between NDs 900A-H), as well as non-transitory machine readable storage media 918 having stored therein networking software 920. During operation, the networking software 920 may be executed by the networking hardware 910 to instantiate a set of one or more networking software instance(s) 922. Each of the networking software instance(s) 922, and that part of the networking hardware 910 that executes that network software instance (be it hardware dedicated to that networking software instance and / or time slices of hardware temporally shared by that networking software instance with others of the networking software instance(s) 922), form a separate virtual network element 930A-R. Each of the virtual network element(s) (VNEs) 930A-R includes a control communication and configuration module 932A-R (sometimes referred to as a local control module or control communication module) and forwarding table(s) 934A-R, such that a given virtual network element (e.g., 930A) includes the control communication and configuration module (e.g., 932A), a set of one or more forwarding table(s) (e.g., 934A), and that portion of the networking hardware 910 that executes the virtual network element (e.g., 930A).

[0108] In an embodiment, software 920 includes code such as MAC mobility component 923, which when executed by networking hardware 910, causes the special -purpose network device 902 to perform operations of one or more embodiments disclosed herein as part of networking software instances 922 (e.g., operations to handle MAC mobility in an EVPN (e.g., to avoid traffic blackholing)).

[0109] The special-purpose network device 902 is often physically and / or logically considered to include: 1) a ND control plane 924 (sometimes referred to as a control plane) comprising the processor(s) 912 that execute the control communication and configuration module(s) 932A-R; and 2) a ND forwarding plane 926 (sometimes referred to as a forwarding plane, a data plane, or a media plane) comprising the forwarding resource(s) 914 that utilize the forwarding table(s) 934A-R and the physical NIs 916. By way of example, where the ND is a router (or is implementing routing functionality), the ND control plane 924 (the processor(s) 912 executing the control communication and configuration module(s) 932A-R) is typically responsible for participating in controlling how data (e.g., packets) is to be routed (e.g., the next hop for the data and the outgoing physical NI for that data) and storing that routing information in the forwarding table(s) 934A-R, and the ND forwarding plane 926 is responsible for receiving that data on thephysical NIs 916 and forwarding that data out the appropriate ones of the physical NIs 916 based on the forwarding table(s) 934A-R.

[0110] Figure 9B illustrates an exemplary way to implement the special-purpose network device 902 according to some embodiments of the invention. Figure 9B shows a special-purpose network device including cards 938 (typically hot pluggable). While in some embodiments the cards 938 are of two types (one or more that operate as the ND forwarding plane 926 (sometimes called line cards), and one or more that operate to implement the ND control plane 924 (sometimes called control cards)), alternative embodiments may combine functionality onto a single card and / or include additional card types (e.g., one additional type of card is called a service card, resource card, or multi- application card). A service card can provide specialized processing (e.g., Layer 4 to Layer 7 services (e.g., firewall, Internet Protocol Security (IPsec), Secure Sockets Layer (SSL) / Transport Layer Security (TLS), Intrusion Detection System (IDS), peer-to-peer (P2P), Voice over IP (VoIP) Session Border Controller, Mobile Wireless Gateways (Gateway General Packet Radio Service (GPRS) Support Node (GGSN), Evolved Packet Core (EPC) Gateway)). By way of example, a service card may be used to terminate IPsec tunnels and execute the attendant authentication and encryption algorithms. These cards are coupled together through one or more interconnect mechanisms illustrated as backplane 936 (e.g., a first full mesh coupling the line cards and a second full mesh coupling all of the cards).

[0111] Returning to Figure 9A, the general purpose network device 904 includes hardware 940 comprising a set of one or more processor(s) 942 (which are often COTS processors) and physical NIs 946, as well as non-transitory machine readable storage media 948 having stored therein software 950. During operation, the processor(s) 942 execute the software 950 to instantiate one or more sets of one or more applications 964A-R. While one embodiment does not implement virtualization, alternative embodiments may use different forms of virtualization. For example, in one such alternative embodiment the virtualization layer 954 represents the kernel of an operating system (or a shim executing on a base operating system) that allows for the creation of multiple instances 962A-R called software containers that may each be used to execute one (or more) of the sets of applications 964A-R; where the multiple software containers (also called virtualization engines, virtual private servers, or jails) are user spaces (typically a virtual memory space) that are separate from each other and separate from the kernel space in which the operating system is run; and where the set of applications running in a given user space, unless explicitly allowed, cannot access the memory of the other processes. In another such alternative embodiment the virtualization layer 954 represents a hypervisor (sometimes referred to as a virtual machine monitor (VMM)) or a hypervisor executing on top of a host operating system, and each of the sets of applications 964 A-R is run on top of a guest operating system within an instance 962A-R calleda virtual machine (which may in some cases be considered a tightly isolated form of software container) that is run on top of the hypervisor - the guest operating system and application may not know they are running on a virtual machine as opposed to running on a “bare metal” host electronic device, or through para-virtualization the operating system and / or application may be aware of the presence of virtualization for optimization purposes. In yet other alternative embodiments, one, some or all of the applications are implemented as unikemel(s), which can be generated by compiling directly with an application only a limited set of libraries (e.g., from a library operating system (LibOS) including drivers / libraries of OS services) that provide the particular OS services needed by the application. As a unikernel can be implemented to run directly on hardware 940, directly on a hypervisor (in which case the unikernel is sometimes described as running within a LibOS virtual machine), or in a software container, embodiments can be implemented fully with unikernels running directly on a hypervisor represented by virtualization layer 954, unikemels running within software containers represented by instances 962A-R, or as a combination of unikernels and the above-described techniques (e.g., unikernels and virtual machines both run directly on a hypervisor, unikernels and sets of applications that are run in different software containers).

[0112] The instantiation of the one or more sets of one or more applications 964A-R, as well as virtualization if implemented, are collectively referred to as software instance(s) 952. Each set of applications 964A-R, corresponding virtualization construct (e.g., instance 962A-R) if implemented, and that part of the hardware 940 that executes them (be it hardware dedicated to that execution and / or time slices of hardware temporally shared), forms a separate virtual network element(s) 960A-R.

[0113] The virtual network element(s) 960A-R perform similar functionality to the virtual network element(s) 930A-R - e.g., similar to the control communication and configuration module(s) 932A and forwarding table(s) 934A (this virtualization of the hardware 940 is sometimes referred to as network function virtualization (NFV)). Thus, NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which could be located in Data centers, NDs, and customer premise equipment (CPE). While embodiments of the invention are illustrated with each instance 962A-R corresponding to one VNE 960A-R, alternative embodiments may implement this correspondence at a finer level granularity (e.g., line card virtual machines virtualize line cards, control card virtual machine virtualize control cards, etc.); it should be understood that the techniques described herein with reference to a correspondence of instances 962A-R to VNEs also apply to embodiments where such a finer level of granularity and / or unikernels are used.

[0114] In certain embodiments, the virtualization layer 954 includes a virtual switch that provides similar forwarding services as a physical Ethernet switch. Specifically, this virtual switch forwards traffic between instances 962A-R and the physical NI(s) 946, as well as optionally between the instances 962A-R; in addition, this virtual switch may enforce network isolation between the VNEs 960A-R that by policy are not permitted to communicate with each other (e.g., by honoring virtual local area networks (VLANs)).

[0115] In an embodiment, software 950 includes code such as MAC mobility component 963, which when executed by processor(s) 942, causes the general purpose network device 904 to perform operations of one or more embodiments described herein as part of software instances 962A-R (e.g., operations to handle MAC mobility in an EVPN (e.g., to avoid traffic blackholing)).

[0116] The third exemplary ND implementation in Figure 9A is a hybrid network device 906, which includes both custom ASICs / special-purpose OS and COTS processors / standard OS in a single ND or a single card within an ND. In certain embodiments of such a hybrid network device, a platform VM (i.e., a VM that that implements the functionality of the special-purpose network device 902) could provide for para- virtualization to the networking hardware present in the hybrid network device 906.

[0117] Regardless of the above exemplary implementations of an ND, when a single one of multiple VNEs implemented by an ND is being considered (e.g., only one of the VNEs is part of a given virtual network) or where only a single VNE is currently being implemented by an ND, the shortened term network element (NE) is sometimes used to refer to that VNE. Also in all of the above exemplary implementations, each of the VNEs (e.g., VNE(s) 930A-R, VNEs 960A-R, and those in the hybrid network device 906) receives data on the physical NIs (e.g., 916, 946) and forwards that data out the appropriate ones of the physical NIs (e.g., 916, 946). For example, a VNE implementing IP router functionality forwards IP packets on the basis of some of the IP header information in the IP packet; where IP header information includes source IP address, destination IP address, source port, destination port (where “source port” and “destination port” refer herein to protocol ports, as opposed to physical ports of a ND), transport protocol (e.g., user datagram protocol (UDP), Transmission Control Protocol (TCP), and differentiated services code point (DSCP) values.

[0118] A network interface (NI) may be physical or virtual; and in the context of IP, an interface address is an IP address assigned to a NI, be it a physical NI or virtual NI. A virtual NI may be associated with a physical NI, with another virtual interface, or stand on its own (e.g., a loopback interface, a point-to-point protocol interface). A NI (physical or virtual) may be numbered (a NI with an IP address) or unnumbered (a NI without an IP address). A loopback interface (and its loopback address) is a specific type of virtual NI (and IP address) of a NE / VNE(physical or virtual) often used for management purposes; where such an IP address is referred to as the nodal loopback address. The IP address(es) assigned to the NI(s) of a ND are referred to as IP addresses of that ND; at a more granular level, the IP address(es) assigned to NI(s) assigned to a NE / VNE implemented on a ND can be referred to as IP addresses of that NE / VNE.

[0119] Some portions of the preceding detailed descriptions have been presented in terms of algorithms and symbolic representations of transactions on data bits within a computer memory. These algorithmic descriptions and representations are the ways used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of transactions leading to a desired result. The transactions are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0120] It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the above discussion, it is appreciated that throughout the description, discussions utilizing terms such as "processing" or "computing" or "calculating" or "determining" or "displaying" or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.

[0121] The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method transactions. The required structure for a variety of these systems will appear from the description above. In addition, embodiments are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of embodiments as described herein.

[0122] An embodiment may be an article of manufacture in which a non-transitory machine- readable storage medium (such as microelectronic memory) has stored thereon instructions (e.g., computer code) which program one or more data processing components (generically referred tohere as a “processor”) to perform the operations described above. In other embodiments, some of these operations might be performed by specific hardware components that contain hardwired logic (e.g., dedicated digital filter blocks and state machines). Those operations might alternatively be performed by any combination of programmed data processing components and fixed hardwired circuit components.

[0123] Throughout the description, embodiments have been presented through flow diagrams. It will be appreciated that the order of transactions and transactions described in these flow diagrams are only intended for illustrative purposes and not intended as a limitation of the present invention. One having ordinary skill in the art would recognize that variations can be made to the flow diagrams without departing from the broader spirit and scope of the invention as set forth in the following claims.

[0124] In the foregoing specification, embodiments have been described with reference to specific exemplary embodiments thereof. It will be evident that various modifications may be made thereto without departing from the broader spirit and scope of the invention as set forth in the following claims. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.

Claims

CLAIMS:

1. A method performed by a network device functioning as a first provider edge (PE) in an ethernet virtual private network (EVPN) to handle media access control (MAC) mobility in the EVPN, the method comprising: receiving (710) a first route advertisement message from a second PE in the EVPN, wherein the first route advertisement message includes a MAC address associated with a host, a first ethernet segment identifier identifying a first ethernet segment, and a sequence number; receiving (720) a second route advertisement message from a third PE in the EVPN, wherein the second route advertisement message includes the MAC address, a second ethernet segment identifier identifying a second ethernet segment, and the same sequence number included in the first route advertisement message; and responsive to a determination that the first route advertisement message and the second route advertisement message include the same MAC address and the same sequence number but include different ethernet segment identifiers, invalidating (730) routes associated with the MAC address having the same sequence number and removing an entry for the MAC address from a MAC table.

2. The method of claim 1, further comprising: receiving (740) traffic destined for the MAC address from a customer edge (CE) attached to the first PE; and sending (750) the traffic destined for the MAC address as broadcast, unknown unicast, or multicast (BUM) traffic in the EVPN using a BUM label due to an absence of the entry for the MAC address in the MAC table.

3. The method of claim 1, further comprising: receiving (760) a third route advertisement message from the third PE, wherein the third route advertisement message includes the MAC address, the second ethernet segment identifier, and a sequence number that is greater than the sequence number included in the first and second route advertisement messages; and responsive to receiving the third address route advertisement message, adding (770) an entry for the MAC address in the MAC table with the third PE as a next hop.

4. The method of claim 1, wherein the first route advertisement message and the second route advertisement message are Multi-Protocol Border Gateway Protocol (MP-BGP) messages including MAC and Internet Protocol (IP) (MAC / IP) advertisement routes.

5. The method of claim 4, wherein the same sequence number is included in MAC mobility extended communities included in the first route advertisement message and the second route advertisement message.

6. A method performed by a network device functioning as a first provider edge (PE) in an ethernet virtual private network (EVPN) to handle media access control (MAC) mobility in the EVPN, the method comprising: sending (810) a first route advertisement message in the EVPN, wherein the first route advertisement message includes a MAC address associated with a host, a first ethernet segment identifier identifying a first ethernet segment, and a sequence number; receiving (820) a second route advertisement message from a second PE in the EVPN, wherein the second route advertisement message includes the MAC address, a second ethernet segment identifier identifying a second ethernet segment, and the same sequence number included in the first route advertisement message; and responsive to a determination that the second route advertisement message includes the same MAC address and sequence number included in the first route advertisement message but a different ethernet segment identifier, sending (830) a withdrawal message for the MAC address in the EVPN, removing an entry for the MAC address from a MAC table, and invalidating routes associated with the MAC address having the same sequence number.

7. The method of claim 6, further comprising: receiving (840) traffic destined for the MAC address from another PE in the EVPN or a local attachment circuit in a same EVPN instance; and flooding (850) the traffic destined for the MAC address to customer edges (CEs) attached to the first PE due to an absence of the entry for the MAC address in the MAC table.

8. The method of claim 6, further comprising: learning (860) about the MAC address via data plane learning; and responsive to the learning, creating (870) an entry for the MAC address in the MAC table and sending a third route advertisement message in the EVPN, wherein the third route advertisement message includes the MAC address, the first ethernet segment identifier, and a sequence number that is greater than the sequencenumber included in the first route advertisement message and the second route advertisement message.

9. The method of claim 6, further comprising: receiving a third route advertisement message from the second PE, wherein the third route advertisement message includes the MAC address, the second ethernet segment identifier, and a sequence number that is greater than the sequence number included in the first and second route advertisement messages; and responsive to receiving the third route advertisement message, adding an entry for the MAC address in the MAC table with the second PE as a next hop.

10. The method of claim 6, wherein the entry for the MAC address is removed from the MAC table based on forcing the entry for the MAC address to age out.

11. The method of claim 6, wherein the first route advertisement message and the second route advertisement message are Multi-Protocol Border Gateway Protocol (MP-BGP) messages including MAC and Internet Protocol (IP) (MAC / IP) advertisement routes.

12. The method of claim 11, wherein the same sequence number is included in MAC mobility extended communities included in the first route advertisement message and the second route advertisement message.

13. A non-transitory machine-readable storage medium that provides instructions that, if executed by a processor of a network device implementing a first provider edge (PE) in an ethernet virtual private network (EVPN), will cause the first PE to carry out the method steps of any one of claims 1-5.

14. A non-transitory machine-readable storage medium that provides instructions that, if executed by a processor of a network device implementing a first provider edge (PE) in an ethernet virtual private network (EVPN), will cause the first PE to carry out the method steps of any one of claims 6-12.

15. A network device to implement a first provider edge (PE) in an ethernet virtual private network (EVPN), the network device comprising: a set of one or more processors; and a non-transitory machine-readable storage medium that provides instructions that, if executed by the set of one or more processors, will cause the first PE to carry out the method steps of any one of claims 1-5.

16. A network device to implement a first provider edge (PE) in an ethernet virtual private network (EVPN), the network device comprising: a set of one or more processors; and a non-transitory machine-readable storage medium that provides instructions that, if executed by the set of one or more processors, will cause the first PE to carry out the method steps of any one of claims 6-12.

Images