Assign unique network addresses to logical network addresses

Abstract

The present disclosure relates to assigning a unique network address to a logical network address. Some embodiments provide a method for a network controller that manages multiple logical networks implemented by multiple managed forwarding elements (MFEs) operating on multiple host machines. The method receives a notification from a specific MFE that an interface corresponding to a logical port of a logical forwarding element is connected to the specific MFE and has a specific logical network address. The method assigns a unique physical network address to the interface. Each of the multiple interfaces connected to the specific MFE is assigned a different physical network address. The method provides the assigned unique physical network address to the specific MFE for the specific MFE to convert a data message sent from the specific logical network address to have a unique physical network address.

Description

[0001] This application is a divisional application based on the patent application with application number 201880043987.1, application date June 27, 2018, and invention name “Assigning a unique network address to a logical network address”. Background Art

[0002] Network virtualization plays a key role in the operation of data centers, where two different approaches are generally used to implement network virtualization. In the overlay approach, the physical data center network is used as a packet carrier, and the network functions (of the logical network) are separated and implemented in the overlay layer above. A common technique is to embed the logical L2 (data link layer) packets in the physical L3 (network layer) packets. In the underlay approach, the physical network devices (e.g., switches, routers) are programmed based on the logical network model so that the physical data center network acts as both a packet carrier and a logical network provider.

[0003] Using an overlay provides flexibility, but for IP networking, the network stack includes five layers instead of three. The additional protocol layers consume additional physical network bandwidth, which means less bandwidth is available for the actual payload. In addition, packet encapsulation and the resulting fragmentation and checksum calculations use additional CPU cycles that would otherwise be available for client workloads (e.g., virtual machines). Thus, other techniques for network virtualization would be useful. Summary of the invention

[0004] Some embodiments provide methods for implementing multiple logical networks in a physical network without encapsulation and without requiring a physical network to perform logical network services and processing. Instead, some embodiments map each logical network address to a unique physical network address and use address substitution instead of encapsulation for logical network packets.

[0005] In some embodiments, a network controller (or a cluster of network controllers) maintains a pool of available physical network addresses and processes requests from a managed forwarding element (MFE) to assign a unique physical address to a logical network address of an interface connected to the MFE. For example, when an interface (e.g., a virtual network interface controller (VNIC)) of a virtual machine (VM) or other data computing node (DCN) corresponding to a logical port of a logical forwarding element is attached to the MFE, the interface is assigned a logical network address. The assignment may be via a dynamic host configuration protocol (DHCP), statically assigned, or otherwise preconfigured, etc. The MFE notifies the network controller of the new logical network address.

[0006] The network controller receives this notification with the logical network address and assigns a unique physical network address to the interface (i.e., maps to the logical network address). The network controller provides the physical network address to the requesting MFE and stores the mapping between the logical network address (and interface) and the physical network address. In some embodiments, based on its network topology information, the network controller distributes the mapping to other MFEs that may be sending packets to the logical network address (or receiving packets from the logical network address) and therefore will require a physical mapping. In other embodiments, when a different MFE receives the first packet sent to the logical network address (e.g., from one of its local DCNs) or sent from the physical network address, the MFE sends a request for a mapping to the controller. The controller notifies the MFE about the mapping so that the MFE can use the mapping to process the packets, as further described below.

[0007] In some embodiments, each physical network address is unique not only within a particular logical network, but also among all logical interfaces of all logical networks implemented within the physical network. That is, although the logical address space can overlap between separate logical networks (i.e., the same subnet and / or IP address can be used in multiple logical networks), the physical network uses a single network address space. In a typical data center, this physical address space is privately allocated (i.e., does not need to be used or known outside the data center), so the available address space is quite large.

[0008] To process a packet at the source MFE (i.e., the MFE that sends the packet to the physical network, which is typically the MFE that first receives the packet from its source DCN), the source MFE first performs logical network processing. This processing may include logically forwarding the packet through one or more logical forwarding elements (e.g., a logical switch, a logical router, and another logical switch), performing logical ACL and distributed firewall checks, etc. If the packet is routed, the time to live and the logical MAC address may be changed.

[0009] Once this logical processing is complete, a typical overlay network will encapsulate packets based on their destination addresses being mapped to physical tunnel endpoint addresses. However, in some embodiments, the MFE determines whether a packet is eligible for address replacement rather than encapsulation. In some embodiments, unicast packets sent only between logical network addresses are eligible for address replacement. That is, multicast / broadcast packets and packets sent to destinations outside the logical network (or received from destinations outside the logical network) are not eligible for address replacement. Assuming that the packet is eligible (and the MFE has mapping information for the source and destination addresses), the source MFE replaces the logical source and destination network (e.g., IP) addresses in the packet with the unique physical addresses to which they are mapped. Some embodiments also modify the source and destination data link (e.g., MAC) addresses using those addresses that will be used to encapsulate the packet (e.g., the source MAC corresponding to the physical interface of the MFE and the destination MAC corresponding to the next hop of the physical network).

[0010] Furthermore, the logical interface may send packets (e.g., ICMP packets) that can cause the physical network routers to perform various unwanted actions when address substitution is used. While encapsulated packets will have this information hidden in the internal header (using, for example, TCP or UDP packet encapsulation), with address substitution, the physical network will see the protocol and may take action on it. Therefore, for certain protocols, the source MFE replaces the protocol header field value with an unused or reserved protocol value that (i) will not cause the physical network to take any unwanted action and (ii) should not be used within the logical network.

[0011] The packet is then processed through the physical network as normal. Once the packet arrives at the destination MFE, additional processing is required to handle the unencapsulated packet. If necessary (for example, if the protocol value is one of the unused or reserved values ​​to which a different value is mapped at the source MFE), the destination MFE maps the protocol field value to its original value. Based on the mapping stored by the MFE, the physical network address is also replaced with the logical network address. To determine the logical data link address, some embodiments use the network topology. If the source and destination network addresses are located on the same logical switch, the data link address will be the address of the corresponding logical interface. However, if the source network address is located on a different logical switch from the destination, the data link address of the logical router interface connected to the logical switch will be the source data link address. Once the data link layer address is also replaced, the MFE can perform any additional required logical processing and deliver the packet to the destination interface.

[0012] The foregoing summary is intended to serve as a brief introduction to some embodiments of the present invention. This is not meant to be an introduction or overview of all the inventive themes disclosed in this document. The following specific embodiments and the drawings referenced in the specific embodiments will further describe the embodiments described in the summary as well as other embodiments. Accordingly, in order to understand all the embodiments described in this document, a comprehensive review of the summary, the specific embodiments, and the drawings is required. In addition, the claimed subject matter will not be limited by the illustrative details in the summary, the specific embodiments, and the drawings, but will be defined by the appended claims, because the claimed subject matter may be implemented in other specific forms without departing from the spirit of the subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The novel features of the invention are set forth in the appended claims. However, for purposes of illustration, several embodiments of the invention are set forth in the following figures.

[0014] Figure 1 The network controller and its communication with the MFE to provide the MFE with the physical IP address of the newly connected interface is conceptually illustrated.

[0015] Figure 2 The process of some embodiments for assigning physical IP addresses to map to logical IP addresses is conceptually illustrated.

[0016] Figure 3 Conceptually illustrated is a process of some embodiments for releasing an assigned physical IP address when a logical interface is moved or released.

[0017] Figure 4 Conceptually illustrates a set of MFEs implementing at least one logical network within a data center network of some embodiments, and the difference between physical network traffic between two logical network endpoints (eg, VMs) and between a logical network endpoint and an external network.

[0018] Figure 5 The process of some embodiments for replacing a logical IP address with a physical IP address is conceptually illustrated.

[0019] Figure 6 The process of some embodiments for replacing a physical IP address with a logical IP address before delivering a packet to an interface is conceptually illustrated.

[0020] Figure 7 A logical network and the logical-to-physical IP address mappings assigned to endpoints of the network are conceptually illustrated.

[0021] Figure 8 and Fig. 9 An example of a packet sent through the physical implementation of the logical network is illustrated.

[0022] Fig.10 Electronic systems that implement some embodiments of the present invention are conceptually illustrated. DETAILED DESCRIPTION

[0023] In the following detailed description of the invention, many details, examples and embodiments of the present invention are set forth and described. However, it will be clear and apparent to those skilled in the art that the present invention is not limited to the embodiments set forth, and the present invention can be practiced without some of the specific details and examples discussed.

[0024] Some embodiments provide methods for implementing multiple logical networks in a physical network without encapsulation and without requiring a physical network to perform logical network services and processing. Instead, some embodiments map each logical network address to a unique physical network address and use address substitution instead of encapsulation for logical network packets.

[0025] In some embodiments, a network controller (or a cluster of network controllers) maintains a pool of available physical network addresses and processes a request from a managed forwarding element (MFE) to assign a unique physical address to a logical network address of an interface connected to the MFE. For example, when an interface (e.g., a virtual network interface controller (VNIC)) of a virtual machine (VM) or other data computing node (DCN) corresponding to a logical port of a logical forwarding element is attached to the MFE, the interface is assigned a logical network address.

[0026] Figure 1 Such a network controller 100 is conceptually illustrated and its communication with the MFE 105 to provide the physical IP address of the newly connected interface to the MFE 105. It should be understood that while a single central network controller 100 is shown, in some embodiments a cluster of such controllers operates to communicate with many MFEs on many host machines.

[0027] As shown, MFE 105 operates on host machine 110, and at least one DCN (in this case VM 115) is attached to MFE 105. In some embodiments, MFE 105 is a virtual switch or other software forwarding element that operates in virtualization software (e.g., a hypervisor) of host machine 110 and is configured by a network control system including network controller 100. In some embodiments, a local controller operates on host machine 110 (e.g., also within the virtualization software). The local controller receives configuration data from network controller 100 and transforms the configuration data from network controller 100 for MFE 105. In some such embodiments, communications between MFE 105 and controller 100 are sent through the local controller.

[0028] The VM 115 is attached to the MFE 105 via a VNIC or similar interface. When the VNIC is attached to the network, it will be assigned a logical network address. In the subsequent discussion, Internet Protocol (IP) addresses will be used, but it should be understood that in different embodiments, these addresses can be other types of network layer addresses. The logical IP address is the address that the VNIC uses to send / receive traffic on the logical network. As further described below, multiple different logical networks can be implemented within a single physical data center network, each of which has its own address space (which can overlap with the address spaces of other logical networks). The MFE implements the logical network based on configuration data received from the network controller.

[0029] The allocation of IP addresses may be accomplished via Dynamic Host Configuration Protocol (DHCP), static allocation, other pre-configuration of IP, etc. When MFE 105 identifies the logical IP address of the new interface (by intercepting DHCP packets, receiving information from the VNIC, processing packets from the VNIC, etc.), MFE 105 notifies the network controller 100 of the new logical network address and interface so that the network controller 100 can allocate a unique physical IP address to the interface (i.e., mapped to the logical network address).

[0030] Figure 2 Conceptually illustrated is a process 200 of some embodiments for allocating physical IP addresses to map to logical IP addresses. Process 200 is performed by a network controller (eg, controller 100) in response to receiving a request for a physical IP address from an MFE (eg, MFE 105).

[0031] As shown, process 200 begins by receiving (at 205) a new logical IP address and corresponding interface from the MFE. Because logical IP addresses are not necessarily unique to logical networks, additional identifiers are required for mapping. Some embodiments use unique VNIC identifiers or unique logical port identifiers. Figure 1 The MFE 105 is illustrated sending a message 120 to the network controller 100 using the logical IP address and interface of the VNIC through which the VM 115 is connected to the MFE 105. As mentioned, the MFE may have been aware of this after the DHCP request, when the VM 115 sends the first packet, etc. In some embodiments, the message 120 from the MFE only needs to identify the existence of the logical interface on the host 110, because the controller 100 already has the corresponding logical IP address assigned to the interface.

[0032] Process 200 allocates (at 210) the available unique physical IP address to the logical IP address / interface combination in response to the request. In some embodiments, each physical network address is unique not only within a specific logical network, but also among all logical interfaces of all logical networks implemented within the physical network. That is, although the logical address space can overlap between separate logical networks (i.e., the same subnet and / or IP address can be used in multiple logical networks), the physical network uses a single network address space. In a typical data center, the physical address space is privately assigned (i.e., it does not need to be used or known outside the data center), so the available address space is quite large. In some embodiments, the data center can use both IPv4 and IPv6 addresses. In such an embodiment, these addresses are assigned separately. That is, when the logical IPv4 address is sent to the controller 100, the controller 100 assigns a unique physical IPv4 address, and when the logical IPv6 address is sent to the controller 100, the controller 100 assigns a unique physical IPv6 address.

[0033] The process 200 then provides (at 215) the assigned unique physical IP address to the requesting MFE. Figure 1 As shown, the network controller 100 sends a message 125 with the assigned physical IP address to the MFE 100. As mentioned, in some embodiments, the message is sent to a local controller on the host 100, which in turn provides the data to the MFE 105. The MFE 105 stores the mapping and uses the mapping to process packets sent to and from the VM 115, as described in more detail below. In some embodiments, the MFE sends a gratuitous ARP packet to notify the physical network of the new IP address.

[0034] Process 200 also stores (at 220) a mapping of logical IP addresses and interfaces to physical IP addresses. Figure 1As shown, the network controller 100 stores a physical to logical network address mapping table 130, as well as a pool 135 of available IP addresses and a waiting pool 140 of IP addresses. The network controller 100 stores the mapping table (which, in some embodiments, also identifies the host machine for each logical IP address and interface combination) in order to distribute the mapping to other MFEs that need the data. In some embodiments, based on its network topology information, the network controller distributes the mapping to other MFEs that may be sending packets to (or receiving packets from) the logical network address and therefore will need a physical mapping. In other embodiments, when a different MFE receives the first packet sent to the logical network address (e.g., from one of its local DCNs) or sent from the physical network address, the MFE sends a request for the mapping to the controller 100. The controller 100 notifies the MFE about the mapping so that the MFE can use the mapping to process the packet, as described further below.

[0035] As mentioned, the network controller 135 also includes a pool 135 of available physical IP addresses and a waiting pool 140 of physical IP addresses. As described above, physical IP addresses are unique within a data center (or other privately assigned physical network). Therefore, the pool 135 of available physical IP addresses lists all IP addresses that can be used for mapping—that is, physical IP addresses that are not currently mapped to the logical IP address of the operating interface. Once the network controller 100 assigns a particular physical IP address to an interface, the controller 100 stores the mapping in the table 130 and removes the physical IP address from the pool 135 of available IPs.

[0036] Figure 3 Conceptually illustrates some embodiments of process 300 for releasing an assigned physical IP address when a logical interface is moved or released. Process 300 is performed by a network controller (eg, controller 100) in response to receiving notification from an MFE (eg, MFE 105) that a logical interface is no longer in use.

[0037] As shown, process 300 begins with receiving (at 305) a notification from the MFE (or a local controller operating on a host with the MFE) that an interface with a logical IP address no longer exists on the MFE. If the VM is migrated to a different host, some embodiments release the physical IP and reallocate a new physical IP; other embodiments maintain the same logical IP to physical IP mapping. Other situations that may cause the logical IP address to no longer exist on the MFE are the removal of the interface from its logical network (i.e., by an administrator changing the logical network configuration), or the logical IP being changed (e.g., also by a change to the logical network configuration).

[0038] In response, the process 300 places (at 310) the physical IP address corresponding to the released logical IP address in a waiting pool for a threshold period of time. As indicated, the network controller 100 includes a waiting pool 140 of physical IP addresses. The waiting pool 140 is used to ensure that the physical IP address is not reassigned too quickly after being released, thereby giving the network time to flush packets that may have been sent to the previous interface to which the physical IP address is mapped.

[0039] Therefore, the process determines (at 315) whether the time period has expired. If not, the process continues to evaluate this until the time period expires. It should be understood that the process 200 (and other processes described herein) is a conceptual process, and some embodiments do not perform continuous checks on each physical IP address in the waiting pool 140. Instead, some embodiments use event-driven processing that simply waits and then takes action when the waiting period expires. Once the time period has expired, the process 300 moves the physical address from the waiting pool (at 320) to the pool of available physical IP addresses. That is, the network controller 100 moves the IP address from the waiting pool 140 to the available IP address pool 135.

[0040] The above description relates to network controller operations that allocate and manage logical to physical IP address mappings. Once these mappings are allocated, packets are sent between MFEs without encapsulation (at least for certain packets that meet certain criteria). Figure 4 A set of MFEs implementing at least one logical network within a data center network 400 of some embodiments is conceptually illustrated. Specifically, the figure illustrates the difference between physical network traffic between two logical network endpoints (eg, VMs) and physical network traffic between a logical network endpoint and an external network.

[0041] As shown, data center 400 includes two host machines 405 and 410 hosting VMs that belong to the same logical network (they can be attached to the same logical switch or different logical switches). VMs 415 and 420 are connected to MFEs 425 and 430, respectively, which operate on host machines 405 and 410 to implement the logical network. In addition, the logical network to which VMs 415 and 420 belong includes a connection to an external network 435 (e.g., a logical router connection). The connection is implemented by a gateway 440 operating on a third host machine 445. In some embodiments, gateway 440 is a separate component of the logical router and can be implemented in a VM or other DCN on host 445, in a data path of host 445, and the like.

[0042] When VM 420 (or VM 415) sends or receives traffic to or from external network 435, the traffic between gateway 440 and MFE 430 is encapsulated using physical IP addresses. As shown by packet 450, the traffic includes an internal IP and Ethernet header and an external (encapsulated) IP and Ethernet header. For simplicity, other internal and external protocols (e.g., transport protocols) are not shown here. Because the external IP address will not have a mapping to a unique IP address, if the MFE or gateway were to replace the IP in the packet (e.g., with the IP address of the PNIC of host 445), the receiving MFE / gateway would not be able to map it back to the correct IP address. Instead, encapsulation is used for such communications between logical network endpoints and external networks in order to preserve these addresses.

[0043] On the other hand, when VM 415 sends packets to VM 420 (or vice versa), MFE 425 performs address replacement to replace the logical IP (and logical MAC) addresses with the physical IP and MAC addresses, as indicated by packet 455. If the network is limited by the maximum transfer size, then packet 455 has fewer headers and therefore has more space for the payload without fragmentation. Address replacement can be used for packet 455 because the service is a unicast communication between two logical network endpoints that have a one-to-one mapping with the physical IP addresses. In some embodiments, the MFE does not use address replacement for multicast / broadcast communications because the packets are sent to multiple physical destinations. However, in other embodiments, at least some multicast / broadcast packets are copied by the MFE into unicast packets (e.g., separate unicast packets for each destination, each packet having a different destination address), and these unicast packets can be sent onto the physical network using address replacement instead of encapsulation.

[0044] Figure 5 and Figure 6 The processes performed by the source MFE (i.e., the first hop MFE of a packet) and the destination MFE (the receiver of such a packet via the physical network) to perform address replacement on the packet are described. These processes assume that the MFE performing the corresponding processes has the logical IP to physical IP mapping information and does not need to request this information from the network controller in order to process the packet.

[0045] The part reference Figure 7-9 describe Figure 5 and Figure 6 processing. Figure 7 The logical network 700 and the logical to physical IP address mappings assigned to the endpoints of the network are conceptually illustrated. Figure 8 and Fig. 9An example of a packet sent through the physical implementation of the logical network is illustrated. Logical network 700 includes two logical switches 705 and 710 logically connected through a logical router 715. Two VMs (VM1 and VM2) are connected to the first logical switch 705, and two VMs (VM3 and VM4) are connected to the second logical switch 710. Each of these logical interfaces has a MAC address (MAC A, MAC B, MAC C, and MAC D). In addition, the logical router downlinks (interfaces to the logical switches) have their own logical MAC addresses (MAC E and MAC F).

[0046] The logical to physical IP address mapping table 720 is information that will be stored by the network controller (or network controller cluster) and the MFE that implements the logical network. As shown in the table, the VMs are implemented on three hosts, so the three MFEs operating on these hosts will store the information in the mapping table 720. VM1 and VM3 are implemented on the first host, with VM2 on the second host and VM4 on the third host. The first logical switch 705 is assigned subnet 10.1.1.0 / 24, and the logical IP addresses of the two VMs on this subnet are 10.1.1.5 and 10.1.1.6. Similarly, the second logical switch 710 is assigned subnet 10.2.1.0 / 24, and the logical IP addresses of the two VMs on this subnet are 10.2.1.5 and 10.2.1.6. According to the mapping table 720, each of these logical interfaces is mapped to a unique physical IP address. Although this example shows only a single logical network, if other logical networks are implemented on the host (or even on some of the hosts), those hosts will also map the logical IP addresses of the additional logical networks to unique physical IP addresses. A single host, for example, can have many mappings of different physical IP addresses to different interfaces of different logical networks for the logical IP address 10.1.1.5.

[0047] Figure 5 Conceptually illustrates some embodiments of a process 500 for replacing a logical IP address with a physical IP address. In some embodiments, the source MFE of the packet (ie, the MFE to which the source interface of the packet is connected) performs the process 500 on the packet upon receiving the packet (eg, from a VNIC).

[0048] As shown, process 500 begins by receiving (at 505) a packet from an interface having a logical IP address. The packet being sent will have logical source and destination IP addresses and logical source and destination MAC addresses. The source address is the address of the interface (e.g., VNIC or similar interface) from which the MFE received the packet. The destination IP address is the address of the final destination of the packet, while the MAC address is either the address of the destination (if the destination is on the same logical switch) or the address of the local logical gateway (if the packet requires logical routing).

[0049] Figure 8 and Fig. 9 Figure 1 shows an example of such packets as they are sent over a physical network. Figure 8 In , VM1 sends packet 800 to VM2 (on the same logical switch, but operating in a different physical host machine). Packet 900 sent to MFE 805 has a source IP address of 10.1.1.5, a destination IP address of 10.1.1.6, a source MAC address of MAC A, and a destination MAC address of MAC B. In addition, the protocol field of the IP header has a value of 17 (for User Datagram Protocol (UDP)). Fig. 9 , VM1 sends packet 900 to VM4 (operating on a different logical switch and in a different physical host machine). Packet 900 sent to MFE 805 has a source IP address of 10.1.1.5, a destination IP address of 10.2.1.6, a source MAC address of MAC A, and a destination MAC address of MAC E (corresponding to the default gateway of VM1). In addition, the protocol field of the IP header has a value of 1 (for Internet Control Message Protocol (ICMP)).

[0050] Return to Figure 5 , process 500 performs (at 510) logical processing on the received packet. That is, the MFE processes the packet through the logical network, which may include the application of ACL and firewall (e.g., distributed firewall) rules, network address translation (NAT) processing, distributed load balancing, etc. Logical processing also includes logical switching and / or routing. If logical routing is required (e.g., for Fig. 9 If the packet is 900), the logical MAC address is modified and the time to live (TTL) of the packet is decremented.

[0051] After the logical processing is complete, the process 500 determines (at 515) whether the packet is eligible for address replacement. In some embodiments, only unicast packets sent between logical network addresses are eligible for address replacement. That is, multicast / broadcast packets and packets sent to (or received from) destinations outside the logical network are not eligible for address replacement. Because the logical IP address is no longer in the packet at all when address replacement is used, some embodiments only use this technique when there is a 1:1 mapping between the logical IP addresses being replaced and the physical IP addresses that replace them.

[0052] In the case of broadcast / multicast, the MFE does not use address replacement because packets are sent to multiple physical destinations. However, in other embodiments, the MFE replicates at least some of the multicast / broadcast traffic into multiple unicast packets, and these unicast packets can be sent onto the physical network using address replacement rather than encapsulation. For packets sent to / from external networks, the use of address replacement would require the assignment of a unique physical IP address to each external IP address that communicates with the logical network. Given the large number of such IP addresses and the more likely transient nature of the communications, the value of such local physical IP address assignments may be small.

[0053] If the packet is not eligible for address replacement (e.g., the packet is a multi-recipient packet, or is addressed to or received from an external IP address that is not a logical network endpoint), the process 500 encapsulates (at 520) the packet. For the encapsulation header, some embodiments use a tunnel endpoint IP address on the physical network but separate from the unique physical IP address used for address replacement. The process 500 then proceeds to 550, as described below.

[0054] On the other hand, when the packet is eligible for address replacement, the process identifies (at 525) the source and destination logical IP addresses and the unique physical IP address of the interface. The source MFE identifies the logical IP address based on data in the packet header field and identifies the source interface based on the interface from which the packet was received. The destination logical interface is identified by the MFE during a logical processing operation (e.g., during logical forwarding).

[0055] The MFE queries its IP address mapping table to identify the physical IP address. In some embodiments, if the MFE does not have a unique physical IP address stored for the destination logical IP address and interface (or source if this is an initial packet from a source interface), the MFE sends a message to the network controller requesting a unique physical IP address. In some embodiments (not shown in this process), rather than waiting for the controller, the first packet (or first few packets) are encapsulated rather than sent using address substitution until the MFE receives the corresponding physical IP address from the network controller.

[0056] However, assuming that the physical IP address is identified, the process 500 replaces (at 530) the logical IP address in the packet with the identified unique physical IP address. In addition, the process modifies (at 532) the time-to-live (TTL) field of the packet to take into account the number of physical network hops that the packet will traverse (each hop will decrement the TTL field). In some embodiments, the TTL field should only be decremented by logical processing (for each logical router that processes the packet). The physical data center network will generally be stable with respect to the number of physical hops between two physical endpoints (when a logical network interface is migrated, this may change the number of physical network hops, but at this time a new unique physical network address will be assigned to the interface). Some embodiments use probe messages or other techniques to determine the number of hops to each possible destination physical IP address and store this information in a mapping table (e.g., as another column in table 720).

[0057] Process 500 also replaces (at 535) the logical MAC address with the physical network MAC address. The source MAC is the MAC of the physical interface corresponding to the source physical IP address, and the destination MAC is the MAC of the local gateway (unless the destination physical interface is on the same physical switch as the source physical interface).

[0058] Figure 8 The diagram shows a packet sent by source MFE 805 with the source and destination physical IP addresses replaced. The source and destination IP addresses are replaced with the unique physical IP addresses shown in mapping table 720 to correspond to 10.1.1.5 (VM1) and 10.1.1.6 (VM2). For the physical MAC addresses, the source MAC (PMAC1) is the MAC of the PNIC corresponding to the 192.168.1.10 address, and the destination MAC (PMAC2) is the MAC of the local default gateway. Fig. 9A similar address replacement of the source and destination IP and MAC addresses of packet 900 is illustrated. The same source physical IP address is used, but the destination IP address corresponding to 10.2.1.6 (VM4) is used. In this case, the same physical MAC address is used as the first packet because the packet is again sent to the local default gateway on the physical network.

[0059] In addition to replacing the logical address with the physical address, the process 500 also determines (at 540) whether the protocol field of the IP header matches one of a set of pre-specified values. When the protocol field does match one of these pre-specified values, the process replaces (at 445) the protocol field value with the replacement value. The logical interface (i.e., the DCN to which the logical interface belongs) may send packets (e.g., ICMP packets) that may cause the physical network routers to perform various unwanted actions when address replacement is used. While the encapsulated packets will have this information hidden in the internal header (using, for example, TCP or UDP packet encapsulation), with address replacement, the physical network will see the protocol and may take action on it. Therefore, for certain protocols, the source MFE replaces the protocol header field value with an unused or reserved protocol value that (i) will not cause the physical network to take any unwanted actions and (ii) should not be used within the logical network.

[0060] For example, Figure 8 The packet 800 has a protocol field value of 17 corresponding to UDP. Since the UDP packet will be forwarded normally by the router of the physical network, the MFE 805 does not modify the protocol field value. Fig. 9 Packet 900 has a protocol field value of 1, which corresponds to ICMP. Physical routers may act on the ICMP packet in an unexpected manner, so MFE 805 replaces it with a value of 143, which is a reserved value that will be ignored by physical network routers.

[0061] Finally, whether the packet is encapsulated or has address replacement performed, the process transmits (at 550) the packet to the physical network (i.e., physical data center network 810). The packet is then processed through the physical network as normal, during which the physical MAC address may be modified.

[0062] Figure 6 Conceptually illustrates some embodiments of replacing a physical IP address with a logical IP address before delivering a packet to an interface. In some embodiments, the packet's destination MFE (i.e., the MFE to which the packet's destination interface is connected) performs process 600 on the packet when it is received from the physical data center network.

[0063] As shown, process 600 begins by receiving (at 605) a logical network packet having a physical IP address. The received packet will have a physical IP address that may correspond to a logical interface or may be a tunnel endpoint address in an encapsulation header. In some embodiments, these physical IP addresses are IP addresses that are either added as an encapsulation header or replaced in the packet by the source MFE (e.g., using a packet such as Figure 5 shown). Figure 8 , packet 800 has the same source and destination physical IP addresses when received by destination MFE 815 as when sent by source MFE 805, but has a different physical MAC address due to routing through physical data center network 810. Fig. 9 This is also the case in the example shown.

[0064] Thus, the process 600 determines (at 610) whether the packet is encapsulated. In some embodiments, the IP address will be different for an encapsulated packet compared to an unencapsulated packet. Specifically, if the source and destination IP addresses correspond to the tunnel endpoints of the source and destination MFEs, the packet is encapsulated. On the other hand, if the source and destination IP addresses are unique physical IP addresses in the MFE's logical to physical IP address mapping table, the packet is sent using address replacement. If the packet is encapsulated, the process decapsulates the packet (at 615) and proceeds to 645 described below. It should be noted that in some embodiments, the MFE performs additional processing to determine that the packet is not sent to an IP address that is not associated with a VTEP and a unique physical IP address mapped to a logical IP address. For example, in some embodiments, the MFE may receive and process management services or other types of services.

[0065] If the packet is not encapsulated (ie, because address replacement is performed on the packet at the source MFE), process 600 performs substantially the same steps as Figure 5 The process 600 determines (at 620) whether the protocol field matches one of a set of pre-specified mapping values. This identifies whether the protocol field value is one of the reserved or unused values ​​to which a particular protocol field value (e.g., ICMP) is mapped. If this is the case, the process replaces (at 625) the protocol field value with the original value. For example, in Fig. 9 In the example, MFE 905 maps the value 143 (reserved value) back to the original value 1 (for ICMP).

[0066] Process 600 identifies (at 630) the logical IP addresses and interfaces for the source and destination physical IP addresses. As mentioned, each physical IP address is mapped not only to a logical IP address, but also to a logical interface. While the source interface is not necessarily critical to the destination MFE (although it may be critical depending on the required processing), the destination interface is important in terms of delivering the packet to the appropriate interface.

[0067] Based on the information identified from the physical IP address, the process 600 replaces (at 635) the physical IP address in the packet with the identified logical IP address. These should be the logical IP addresses that were in the packet before the source MFE performed the address replacement. In addition, the process replaces (at 640) the physical MAC address with the logical MAC address based on the logical network topology. If the source interface and the destination interface are on the same logical switch, the MAC address will be the MAC address corresponding to those interfaces. However, if the source interface is on a different logical switch than the destination interface, the MAC address of the logical router interface connected to the destination logical switch will be the source MAC address.

[0068] exist Figure 8 , MFE 815 translates the source and destination IP addresses back to 10.1.1.5 and 10.1.1.6, respectively. Similarly, because the source and destination interfaces (VM1 and VM2) are on the same logical switch 705, the source and destination logical MAC addresses are both the MAC addresses corresponding to the interfaces (i.e., the same as when the packet was sent to MFE 805). However, in Fig. 9 , the source logical MAC address in the packet 900 sent from the MFE 905 to the destination VM4 is MAC F, i.e., the address of the logical router interface connected to the logical switch 710. In addition, the destination logical MAC address of the packet is MAC D, i.e., the MAC address of the destination VM4. The MFE 905 recognizes that the source interface is located on a different logical switch 705 based on the network topology, and performs the MAC address replacement.

[0069] After reverse address replacement has been completed (or the packet has been decapsulated), process 600 performs (at 645) any additional logical processing, such as applying egress ACL rules, additional distributed firewall rules, etc. The process then delivers (at 650) the packet to the identified destination interface.

[0070] Many of the features and applications described above are implemented as software processes that are specified as sets of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more processing units (e.g., one or more processors, cores of processors, or other processing units), they cause the processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, RAM chips, hard drives, EPROMs, etc. Computer-readable media do not include carrier waves and electronic signals transmitted wirelessly or through wired connections.

[0071] In this specification, the term "software" is intended to include firmware residing in a read-only memory or an application stored in a magnetic storage device, which can be read into a memory for processing by a processor. Moreover, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while retaining different software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that implement the software inventions described herein together are within the scope of the present invention. In some embodiments, the software program defines one or more specific machine implementations that execute and perform the operations of the software program when installed to operate on one or more electronic systems.

[0072] Fig.10 An electronic system 1000 is conceptually illustrated to implement some embodiments of the present invention. The electronic system 1000 can be used to perform any of the above-mentioned control, virtualization or operating system applications. The electronic system 1000 can be a computer (e.g., a desktop computer, a personal computer, a tablet computer, a server computer, a mainframe, a blade computer, etc.), a phone, a PDA, or any other type of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1000 includes a bus 1005, a processing unit 1010, a system memory 1025, a read-only memory 1030, a permanent storage device 1035, an input device 1040, and an output device 1045.

[0073] Bus 1005 collectively represents all system buses, peripheral buses, and chipset buses that communicatively connect the many internal devices of electronic system 1000. For example, bus 1005 communicatively connects processing unit 1010 with read-only memory 1030, system memory 1025, and permanent storage device 1035.

[0074] From these various memory units, processing unit 1010 retrieves instructions to execute and data to process in order to perform the processes of the present invention. In different embodiments, the processing unit may be a single processor or a multi-core processor.

[0075] The read-only memory (ROM) 1030 stores static data and instructions required by the processing unit 1010 and other modules of the electronic system. On the other hand, the permanent storage device 1035 is a read-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the electronic system 1000 is turned off. Some embodiments of the present invention use a mass storage device (such as a magnetic disk or optical disk and its corresponding disk drive) as the permanent storage device 1035.

[0076] Other embodiments use removable storage devices (such as floppy disks, flash drives, etc.) as permanent storage devices. Like permanent storage device 1035, system memory 1025 is a read-write memory device. However, unlike storage device 1035, system memory is a volatile read-write memory, such as random access memory. System memory stores some of the instructions and data that the processor needs at run time. In some embodiments, the processes of the present invention are stored in system memory 1025, permanent storage device 1035, and / or read-only memory 1030. From these various memory units, processing unit 1010 retrieves instructions to be executed and data to be processed in order to perform the processes of some embodiments.

[0077] Bus 1005 is also connected to input and output devices 1040 and 1045. Input devices enable a user to communicate information and select commands to the electronic system. Input device 1040 includes an alphanumeric keyboard and a pointing device (also referred to as a "cursor control device"). Output device 1045 displays images generated by the electronic system. Output devices include printers and display devices, such as cathode ray tubes (CRTs) or liquid crystal displays (LCDs). Some embodiments include devices that function as both input devices and output devices, such as touch screens.

[0078] Finally, if Fig.10 As shown, bus 1005 also couples electronic system 1000 to a network 1065 via a network adapter (not shown). In this manner, the computer may be part of a network of computers, such as a local area network ("LAN"), a wide area network ("WAN"), or an intranet, or a network of networks, such as the Internet. Any or all of the components of electronic system 1000 may be used in conjunction with the present invention.

[0079] Some embodiments include electronic components, such as microprocessors, storage devices, and memories, which store computer program instructions in machine-readable or computer-readable media (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, compact disk-read only (CD-ROM), compact disk-recordable (CD-R), compact disk-rewritable (CD-RW), read-only digital versatile disks (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini SD card, micro SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable A computer readable medium may store a computer program executable by at least one processing unit and including a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as code produced by a compiler, and files including higher-level code that is executed by a computer, electronic component, or microprocessor using an interpreter.

[0080] Although the above discussion mainly involves microprocessors or multi-core processors that execute software, some embodiments are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves.

[0081] As used in this specification, the terms "computer," "server," "processor," and "memory" all refer to electronic or other technical devices. These terms do not include people or groups of people. For the purposes of this specification, the terms "display" or "being displayed" mean displaying on an electronic device. As used in this specification, the terms "computer-readable medium," "computer-readable media," and "machine-readable medium" are collectively limited to tangible, physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, and any other transient signals.

[0082] This specification relates throughout to computing and network environments including virtual machines (VMs). However, a virtual machine is only one example of a data computing node (DCN) or a data computing end node (also referred to as an addressable node). A DCN may include non-virtualized physical hosts, virtual machines, containers that run on top of a host operating system without a hypervisor or a separate operating system, and a hypervisor kernel network interface module.

[0083] In some embodiments, VMs operate on the host with their own client operating systems using the resources of the host virtualized by virtualization software (e.g., hypervisor, virtual machine monitor, etc.). Tenants (i.e., owners of VMs) can choose which application to operate on top of the client operating system. On the other hand, some containers are constructed to run on top of the host operating system without the need for a hypervisor or a separate client operating system. In some embodiments, the host operating system uses namespaces to isolate containers from each other, and thus provides operating system-level separation of different application groups operating in different containers. This separation is similar to the VM separation provided in an environment of hypervisor virtualization of virtualized system hardware, and can therefore be viewed as a form of virtualization that isolates different application groups operating in different containers. Such containers are lighter than VMs.

[0084] In some embodiments, the hypervisor kernel network interface module is a non-VM DCN that includes a network stack with a hypervisor kernel network interface and receive / transmit threads. An example of a hypervisor kernel network interface module is the vmknic module, which is a VMware ESXi TM part of the management program.

[0085] It should be understood that although this specification relates to VMs, the examples given may be any type of DCN, including physical hosts, VMs, non-VM containers, and hypervisor kernel network interface modules. In fact, in some embodiments, the example network may include a combination of different types of DCNs.

[0086] Although the present invention has been described with reference to many specific details, those skilled in the art will recognize that the present invention may be implemented in other specific forms without departing from the spirit of the present invention. Figure 2 , 3 , 5 and 6) conceptually illustrate the processing. The specific operations of these processes may not be performed in the exact order shown and described. The specific operations may not be performed in a continuous sequence of operations, and different specific operations may be performed in different embodiments. In addition, the processing can be implemented using several sub-processes, or as part of a larger macro process. Therefore, it will be understood by those skilled in the art that the present invention will not be limited to the foregoing illustrative details, but will be defined by the appended claims.

Claims

1. A method for a network controller, the network controller managing a plurality of logical networks implemented by a plurality of managed forwarding elements (MFEs) operating on a plurality of host machines in a data center, the method comprising: receiving, from a specific MFE, a notification that an interface corresponding to a logical port of a logical forwarding element is connected to the specific MFE and has a specific logical network address; assigning to the interface a physical network address that is unique within a physical network of the data center, wherein each of a plurality of interfaces connected to the specific MFE is assigned a different physical network address that is unique within the physical network of the data center; and The allocated unique physical network address is provided to the specific MFE so that the specific MFE replaces the specific logical network address with the unique physical network address in data messages sent from the specific logical network address to other logical network destinations within the data center.

2. The method of claim 1, wherein: The interface is a virtual interface of a data computing node DCN operating on the same host machine as the specific MFE.

3. The method of claim 2, wherein: A plurality of DCNs operate on the same host machine as the specific MFE, wherein each of the DCNs has at least one interface connected to the specific MFE, and the method further comprises assigning a unique physical network address within a physical network of the data center to each of the interfaces.

4. The method of claim 3, wherein: The specific logical network address is a first logical network address, the virtual interface is a first virtual interface, and the logical forwarding element is a first logical forwarding element of the first logical network, wherein a second virtual interface of a second DCN connected to a second logical forwarding element of a second logical network also has the specific logical network address, wherein different physical network addresses are assigned to the first virtual interface and the second virtual interface.

5. The method of claim 2, wherein: The specific logical network address is a first logical network address, and the unique physical network address is a first physical network address, wherein the DCN includes at least a second virtual interface connected to the MFE and having a second logical network address, and the method further includes assigning a second physical network address that is unique within the physical network of the data center to the second virtual interface.

6. The method of claim 1, wherein: The specific MFE is a first MFE, the interface is a first interface, the logical port is a first logical port, the logical network address is a first logical network address, and the unique physical network address is a first physical network address, and the method further includes: receiving, from the second MFE, a notification that a second interface corresponding to the second logical port of the logical forwarding element is connected to the second MFE and has a second logical network address; assigning a second physical network address to the second interface; and The allocated second physical network address is provided to the second MFE for the second MFE to replace the second logical network address with the second physical network address in data messages sent from the second logical network address to other logical network destinations within the data center.

7. The method of claim 6, wherein: The first logical network address and the second logical network address are located in the same first subnet, wherein the first physical network address and the second physical network address are located in different second and third subnets.

8. The method of claim 1, wherein: The specific MFE also replaces the unique physical address with the specific logical network address in a data message sent to the unique physical address.

9. The method of claim 1, wherein: The specific MFE is a first MFE, and the method further includes: receiving a request from the second MFE when the second MFE has received a data message having the specific logical network address as a destination address; and The unique physical network address is provided to the second MFE, so that the second MFE uses the unique physical network address to replace the specific logical network address as a destination address of at least a subsequent data message.

10. The method of claim 1, wherein: The specific logical network address and the unique physical network address are Internet Protocol IP addresses, and the method further includes allocating a physical Media Access Control MAC address to the interface.

11. The method of claim 1, wherein: The unique physical network address is allocated from a pool of physical network addresses.

12. The method of claim 1, further comprising: receiving, from the specific MFE, a notification that the interface is no longer connected to the specific MFE; assigning the unique physical network address to a waiting pool for a period of time; and After the time period, the unique physical network address is assigned to a pool of available physical network addresses.

13. A machine-readable medium storing a program, which, when executed by at least one processing unit, implements the method according to any one of claims 1 to 12.

14. An electronic device comprising: a set of processing units; as well as A machine-readable medium storing a program which, when executed by at least one of the processing units, implements the method of any one of claims 1-12.

15. A system comprising means for implementing the method of any one of claims 1-12.

16. A computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 12.

Images