Specifying and utilizing paths through a network

Abstract

The present disclosure relates to specifying and utilizing paths through a network. Traffic engineering refers to the process by which a network manager defines a specific path through a network for a series of data message flows. Methods used to date include MPLS (Multi-Protocol Label Switching) technology, which adds path description information between the Layer 2 and Layer 3 headers. Due to this location of the path description, MPLS is often referred to as a Layer 2.5 protocol. However, MPLS technology and other previous traffic engineering technologies do not easily support encapsulating tenant identifiers. Binding these existing solutions to tenants would require additional strategies and multiple encapsulations for the overlay and underlay layers.

Description

[0001] This application is a divisional application of the invention patent application with application date of October 23, 2018, application number 201880061074.2, and invention name “Specifying and utilizing paths through a network”. Technical Field

[0002] The present disclosure relates generally to network technology and, more particularly, to technology for specifying and utilizing paths through a network. Background Art

[0003] Traffic engineering refers to the process by which network management programs define a specific path through the network for a series of data message flows. Methods used to date include MPLS (Multi-Protocol Label Switching) technology, which adds path description information between the Layer 2 and Layer 3 headers. Due to this location of the path description, MPLS is often referred to as a Layer 2.5 protocol. However, MPLS technology and other previous traffic engineering technologies do not easily support encapsulating tenant identifiers. Binding these existing solutions to tenants would require additional policies and multiple encapsulations for the overlay and underlay layers. Summary of the invention

[0004] For a multi-tenant environment, some embodiments of the present invention provide a novel method for (1) embedding a specific path of a tenant data message flow through a network into a tunnel header of an encapsulated data message flow, and then (2) using the embedded path information to direct the data message flow through the network. In some embodiments, the encapsulating tunnel header is placed in front of (i.e., outside of) the layer 2-4 header of each data message in the flow.

[0005] In some embodiments, the method selects a specific path for the data message flow from two or more feasible such paths through the network. In order to select a path from a set of feasible paths between the source and destination of the data message flow, in some embodiments, the method performs a classification operation on the data message flow. In some embodiments, the classification operation can be based on the layer 2-4 header values ​​of the data message flow (e.g., five tuple identifiers, i.e., protocol, source port and destination port, and source IP address and destination IP address) and / or its associated context attributes, such as its traffic type (i.e., the type of content carried in the data message flow), QoS (quality of service) level, layer 7 parameters, process identifier, user identifier, group identifier, etc.

[0006] In some of these embodiments, the context attributes are captured from a source machine (e.g., a virtual machine, a container, etc.) of the data message flow, and / or captured on a host computer on which the source machine executes. When the set of context attributes includes a traffic type of the flow, the method provides a novel way to select a traffic engineering path for the data message flow, embed information about the path into a tunnel header of the data message flow, and use the information to direct the data message flow through a network.

[0007] After selecting a path for the data message flow, in some embodiments, the method embeds the identified path in a tunnel header that it uses to encapsulate the data message of the flow. Similarly, in some embodiments, the method embeds a tenant identifier (e.g., embeds a VNI, virtual network identifier) ​​in the tunnel header to specify that the data message is associated with a specific tenant (e.g., is sent from a specific tenant's machine). The method sends the encapsulated message along the tunnel to a first intermediate forwarding element, which serves as a first hop along the selected path.

[0008] Some embodiments use the Generic Network Virtualization Encapsulation (Geneve) tunneling protocol to carry path information (e.g., next hop index and hop list) and tenant identifiers. In some embodiments, the tenant identifier is embedded in the Geneve base header, and the path information is embedded in the option TLV (type, length, value) of the Geneve tunnel header. The embedding of the tenant identifier allows data message flows of multiple tenants to use the same tunnel between two hops.

[0009] In some embodiments, the method identifies the selected path by identifying and embedding a list of network addresses (e.g., destination IP addresses) of each hop along the path (e.g., intermediate forwarding elements), and embeds the path into a tunnel header. Furthermore, in some embodiments, the method embeds a hop index value in the tunnel header that identifies one of the embedded network addresses as the network address of the "next" hop for identifying the next hop along the path. As described below, when the hops along the path are embedded in the tunnel header in reverse order (the first hop appears last in the hop list and the last hop appears first in the hop list), each hop along the path decrements the hop index value.

[0010] In some embodiments, the hops along the selected path are connected to each other through a set of tunnels. In some embodiments, some or all of the hops are software forwarding elements (e.g., software switches or routers) executed on a host computer. In some embodiments, one or more of the hops are hardware forwarding elements, such as a top-of-rack switch, a hardware router, etc.

[0011] Thus, after selecting a path for the data message and encapsulating the data message with a tunnel header containing the network addresses of hops along the path, in some embodiments the method passes the encapsulated data message along the tunnel to a first hop along the path (e.g., a first SFE) after a forwarding element connected to the source machine (e.g., after a software forwarding element executing on the same host computer as the source machine).

[0012] For each tunnel connecting two hops along the path, in some embodiments, the encapsulating tunnel header for each message includes an outer header that includes the source and destination network addresses. In some embodiments, the source address of the outer header is the network address (e.g., IP address) of the hop that sent the message along the tunnel, and the destination address is the network address of the hop that received the message along the tunnel. At each particular hop after the forwarding element of the source machine, the particular hop checks the information embedded in the tunnel header to determine that the tunnel header identifies it as the next hop. In some embodiments, the particular hop makes this determination by extracting the next hop index from the tunnel header of the received message, using the index to retrieve the network address of the next hop, and then determining that the network address is its address.

[0013] After determining that the received data message identifies it as the next hop, the particular hop determines whether it is designated as the last hop of the specified path. If so, the particular hop checks the destination network address embedded in the header of the data message. The destination network address (e.g., IP address) specifies the machine (e.g., a standalone computer, a virtual machine, or a container) connected to the hop. Therefore, the particular hop provides the decapsulated data message to the machine.

[0014] On the other hand, when the particular hop determines that it is not designated as the last hop along the path embedded in the tunnel header of the received data message, the particular hop forwards the encapsulated data message along another tunnel to another hop after adjusting the tunnel header parameters. Specifically, to forward the data message to the other hop, in some embodiments, the particular hop (1) decrements the next hop index (extracted from the tunnel header of the received data message) to identify the network address of the other hop as the network address of the next hop along the path, (2) resolves the network address of the next hop to an underlying tunnel that starts at the particular hop and ends at the next hop, and (3) embeds the decremented hop index value in the tunnel header together with the tenant identifier and the list of network addresses of each hop along the path. In some embodiments, the particular hop embeds the network address of the particular hop as the source network address of the encapsulated data information and the network address of the other hop as the destination network address of the encapsulated data information in the outer portion of the tunnel header of the data message.

[0015] Instead of adjusting the tunnel header parameters, in other embodiments, a particular hop decapsulates (i.e., removes) the tunnel header from the received data message and recapsulates the data message with a new tunnel header. One of ordinary skill will recognize that the above-described method for processing embedded path information is merely one way to process such information. Other embodiments process this information in different ways. For example, other embodiments do not store the next hop index in the tunnel header because each intermediate hop is configured to select a next hop address (e.g., a first hop address) and remove its network address from the list of path hop network addresses before passing the encapsulated data message to the next hop along the path. Moreover, the above-described method is used in some embodiments to embed the selected path in a single tenant environment. Therefore, one of ordinary skill in the art will recognize that some embodiments of the present invention are equally applicable to a single tenant data center.

[0016] The foregoing summary is intended to serve as a brief introduction to some embodiments of the present invention. This is not meant to be an introduction or overview of all inventive subject matter disclosed in this document. The following detailed description and the drawings mentioned in the detailed description will further describe the embodiments described in the summary of the invention as well as other embodiments. Therefore, in order to understand all embodiments described by this document, a comprehensive review of the summary of the invention, the detailed description, the drawings and the claims is required. Moreover, the claimed subject matter is not limited by the illustrative details in the summary of the invention, the detailed description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The novel features of the invention are set forth in the appended claims. However, for purposes of illustration, several embodiments of the invention are set forth in the following figures.

[0018] Figure 1 An example of specifying and using a specially defined path according to some embodiments of the present invention is illustrated.

[0019] Figure 2 The Geneve tunnel header of some embodiments is illustrated.

[0020] Figure 3 The Geneve base header of some embodiments is illustrated.

[0021] Figure 4 The processing performed by an entry jump in some embodiments is conceptually illustrated.

[0022] Figure 5 The diagram illustrates the processing performed by a hop along the path when it receives a Geneve encapsulated data message.

[0023] Figure 6A host computer is illustrated for executing the virtual machines, service machines, and path processors of some embodiments in some embodiments.

[0024] Figure 7 An example of how path processors may be managed in some embodiments is illustrated.

[0025] Figure 8 A computer system for implementing some embodiments of the invention is conceptually illustrated. DETAILED DESCRIPTION

[0026] In the following detailed description of the present invention, many details, examples and embodiments of the present invention are set forth and described. However, it will be clear and apparent to those skilled in the art that the present invention is not limited to the embodiments set forth, and the present invention can be practiced without some of the specific details and examples discussed.

[0027] For a multi-tenant environment, some embodiments of the present invention provide a novel method for (1) embedding a specific path of a tenant data message flow through a network in a tunnel header of an encapsulated data message flow, and then (2) using the embedded path information to direct the data message flow through the network. In some embodiments, the encapsulating tunnel header is placed in front of (i.e., outside of) the layer 2-4 header of each data message in the flow.

[0028] In some embodiments, the method selects a particular path for a data message flow from two or more feasible such paths through a network. In order to select a path from a set of feasible paths between a source and a destination of a data message flow, in some embodiments, the method performs a classification operation on the data message flow. In some embodiments, the classification operation can be based on layer 2-4 header values ​​of the data message flow (e.g., five tuple identifiers, i.e., protocol, source port and destination port, and source IP address and destination IP address) and / or its associated context attributes, such as its traffic type (i.e., the type of content carried in the data message flow), QoS level, layer 7 parameters, process identifier, user identifier, group identifier, etc. When the set of context attributes includes the traffic type of the flow, the method provides a novel way to select a traffic engineering path for a data message flow, embed information about the path into a tunnel header of the data message flow, and use the information to guide the data message flow through the network.

[0029] After selecting a path for the data message flow, in some embodiments, the method embeds the identified path in a tunnel header that it uses to encapsulate the data message of the flow. Similarly, in some embodiments, the method embeds a tenant identifier (e.g., embeds a VNI, virtual network identifier) ​​in the tunnel header to specify that the data message is associated with a specific tenant (e.g., is sent from a specific tenant's machine). The method sends the encapsulated message along the tunnel to a first intermediate forwarding element, which serves as a first hop along the selected path.

[0030] In some embodiments, the method identifies the selected path by identifying and embedding a list of network addresses (e.g., destination IP addresses) of each hop along the path (e.g., intermediate forwarding elements), and embeds the path into a tunnel header. Furthermore, in some embodiments, the method embeds a hop index value in the tunnel header that identifies one of the embedded network addresses as the network address of the "next" hop for identifying the next hop along the path. As described below, when the hops along the path are embedded in the tunnel header in reverse order (the first hop appears last in the hop list and the last hop appears first in the hop list), each hop along the path decrements the hop index value.

[0031] In some embodiments, the hops along the selected path are connected to each other through a set of tunnels. In some embodiments, some or all of the hops include software forwarding elements (SFEs) executed on a host computer. In some embodiments, examples of such SFEs include software routers and switches. In some embodiments, one or more of the hops are hardware forwarding elements, such as top-of-rack switches, hardware routers, etc.

[0032] After selecting a path for the data message and encapsulating the data message with a tunnel header that includes the network addresses of the hops along the path, in some embodiments, the method passes the encapsulated data message along the tunnel to a first hop along the path (e.g., a first SFE) after a forwarding element connected to the source machine (e.g., after a software forwarding element executing on the same host computer as the source machine). The outer portion of the tunnel header includes the source and destination network addresses. At the ingress hop, the source address of the outer portion is the network address (e.g., IP address) of the ingress hop (or the virtual tunnel endpoint (VTEP) used by that hop) that originally sent the message along the tunnel, and the destination address is the network address of the first hop (or its VTEP) that received the message along the tunnel.

[0033] As used in this document, a data message refers to a collection of bits in a specific format that is sent across a network. One of ordinary skill in the art will recognize that the term "data message" is used in this document to refer to a collection of bits in various formats that can be sent across a network, such as Ethernet frames, IP packets, TCP segments, UDP datagrams, etc. Similarly, as used in this document, references to L2, L3, L4, and L7 layers (or Layer 2, Layer 3, Layer 4, Layer 7) are references to the second data link layer, the third network layer, the fourth transport layer, and the seventh application layer of the OSI (Open Systems Interconnection) layer model, respectively.

[0034] Figure 1 An example of specifying and using traffic engineering paths according to some embodiments of the present invention is illustrated. In this example, two paths 180 and 185 are specifically specified in the network between two guest virtual machines (GVMs) 102 and 106 executed on two hosts 104 and 108 in a multi-tenant data center. The GVM belongs to a tenant that is associated with a specific virtual network identifier VNI in the data center. Even in Figure 1 In the example shown, the embedded path information allows the data message flow to take a specific path through one data center, but those skilled in the art will appreciate that in other embodiments, the path information is used to specify a path between machines in two or more data centers of the same enterprise, the same public cloud provider, and / or different public cloud providers. Similarly, in some embodiments, the path information can be used to specify a path between machines located in one or more private data centers or branches and one or more public cloud data centers.

[0035] Each path 180 or 185 begins at an ingress SFE (e.g., a software switch or router) 120 executing on a host 104 and terminates at an egress SFE 128 executing on a host 108. These SFEs 120 and 128 serve as the ingress and egress hops for each path. Each path also has one or more intermediate hops, where path 180 has intermediate hops 122 and 124, and path 185 has intermediate hop 126. Each path is formed by a series of tunnels 150-158 established between adjacent hops along the path. In this example, each hop includes a software forwarding element executing on a host computer, but this is not necessarily the case. For example, in some embodiments, one or more of the intermediate hops may be a top-of-rack switch, a backbone switch, a hardware router, or any other type of forwarding element.

[0036] exist Figure 1In the example shown, path 180 is selected for a data message flow from GVM 102 to GVM 106. As shown, the ingress and intermediate hops pass the data message along the path through tunnels 150, 152, and 154 with an encapsulating tunnel header 140 placed in front of (i.e., outside of) the layer 2-4 header 130 of the data message 100. In addition, as the encapsulated data message is sent along each tunnel, an outer portion of the tunnel header (not shown) identifies the IP addresses of the source and destination endpoints of the tunnel as the source and destination IP addresses of the encapsulated data message.

[0037] In some embodiments, the source and destination tunnel endpoints are the VTEPs associated with the two hop endpoints of the tunnel. For example, for tunnel 150, the outer portion of tunnel header 140 identifies the IP address of the VTEP of ingress hop 120 as the source IP address of the encapsulated data message, and identifies the IP address of the VTEP of hop 122 as the destination IP address of the encapsulated data message. In the encapsulated data message forwarded by ingress hop 120 or intermediate hops 122 or 124, tunnel header 140 contains the tenant's VNI and a path description that specifies the remainder of the path at that hop. As shown, in some embodiments, the path description is expressed in terms of a list of IP addresses of the remaining hops.

[0038] In some embodiments, ingress hop 120 includes an SFE that receives data messages from source GVM 102, and a path processing filter executed on the same host computer 104. As described further below, a hook (e.g., a callback function) is defined in the port of the SFE to intercept the data message sent by the source GVM 102 and forward the data message to the path processing filter. The filter then performs a classification operation on the data message in order to select a path 180 for the data message to its destination GVM 106.

[0039] The classification operation selects path 180 from a group of two candidate paths 180 and 185, which are feasible candidate paths that have been specifically defined (e.g., by a controller or forwarding element) to reach the destination GVM 106. In some embodiments, the classification operation is based on a set of attributes associated with the data message flow. In some embodiments, the set of attributes includes only layer 2-4 header values ​​of the data message flow. However, in other embodiments, the set of attributes includes contextual attributes related to the data message flow, such as the traffic type of the data message flow (i.e., the type of content carried in the data message flow), QoS level, layer 7 parameters, process identifiers, user identifiers, group identifiers, etc.

[0040] When the set of context attributes includes the traffic type of the flow, the ingress hop selects a traffic engineering path for the data message flow, embeds information about the path into a tunnel header of the data message flow, and uses the information to direct the data message flow through the network. For example, based on deep packet inspection performed on host computer 104, a first data message flow between GVMs 102 and 106 may be identified as carrying video conferencing data for corporate executives, while a second data message flow between the two GVMs may be identified as carrying chat message data between two non-executives. Based on the context data, path 180 is selected for the first message flow because the path may be better for latency or waiting time, while path 185 is selected for the second message flow because the path may be better for cost.

[0041] After selecting a path for the intercepted data message, the path processing filter of the ingress hop, in some embodiments, embeds the selected path into the tunnel header 140 that it uses to encapsulate the data message 100. As shown, in some embodiments, the path processing filter identifies and embeds the selected path based on the IP addresses of the intermediate hops 122 and 124 and the egress hop 128. In addition to the IP addresses of these hops, the ingress path processing filter also embeds a tenant identifier (in this case, the VNI of a specific tenant) in the tunnel header to specify that the data message 100 is associated with a specific tenant (e.g., was sent from a specific tenant's GVM).

[0042] In some embodiments, the ingress path processing filter embeds the network address of the endpoint of the first tunnel 150 in the outer portion of the tunnel header 140. In some embodiments, the path processing filter also embeds a next hop value in the tunnel header that identifies the embedded network address of the intermediate hop 122 as the network address of the “next hop.” As described further below, in some embodiments, subsequent hops along the path use the index to identify the next hop along the path.

[0043] After selecting a path for data message 100 and encapsulating the data message using tunnel header 140, ingress hop 120 passes the encapsulated data message along tunnel 150 to intermediate hop 122. As described above, in some embodiments, the outer portion of tunnel header 140 identifies the IP address of the VTEP of ingress hop 120 as the source IP address of the encapsulated data message and the IP address of the VTEP of first hop 122 as the destination IP address of the encapsulated data message.

[0044] At each subsequent hop 122, 124, or 128 along path 180, the hop examines the tunnel header and determines that the received data message is addressed to it. In some embodiments, each particular hop makes this determination by extracting a hop index from the tunnel, using the index to retrieve the network address of the next hop from the tunnel header, and then determining that address to be its network address. After determining that the received data message identifies it as the next hop, the particular hop determines whether it is designated as the last hop of the designated path.

[0045] If not, then the particular hop forwards the encapsulated data message to another hop along the other tunnel after adjusting the tunnel header parameters. Specifically, to forward the data message to another hop, in some embodiments, the particular hop (1) decrements a next hop index (extracted from the tunnel header of the received data message) to identify the network address of the other hop as the network address of the next hop along the path, (2) resolves the network address of the next hop to an underlying tunnel that starts at the particular hop and ends at the next hop, and (3) embeds the decremented hop index value in the tunnel header along with the tenant identifier and a list of network addresses of each hop along the path.

[0046] Figure 1 The adjustment of the hop index is illustrated by showing it as (1) referencing the IP address of hop 122 initially when the encapsulated data message leaves the ingress hop, (2) referencing the IP address of hop 124 when the encapsulated data message leaves hop 122, and (3) referencing the IP address of hop 128 when the encapsulated data message leaves hop 124. Each particular hop also adjusts the outer portion of the tunnel header so that the network address of the particular hop is the source network address of the encapsulated data message, and the network address of the subsequent hop along path 180 is the destination network address of the encapsulated data message. For example, hop 122 places its IP address as the source IP address of the outer header and places the IP address of hop 124 as the destination IP address in the header. Similarly, hop 124 places its IP address as the source IP address of the outer header and places the IP address of egress hop 128 as the destination IP address in the header.

[0047] In other embodiments, instead of adjusting the tunnel header parameters, a particular hop decapsulates (i.e., removes) the tunnel header from the received data message and recapsulates the data message with a new tunnel header. Additionally, other embodiments do not store the next hop index in the tunnel header because, in these embodiments, each intermediate hop is configured to select a next hop address (e.g., a first hop address) and remove its network address from the list of path hop network addresses embedded in the tunnel before passing the encapsulated data message to the next hop along the path.

[0048] When the encapsulated data message reaches egress hop 128, the hop determines that it is designated as the last hop along the path embedded in the tunnel header of the data message (e.g., determines that the next hop index is its minimum value and it identifies the address of the hop). Therefore, it removes the tunnel header and examines the destination network address embedded in the data message header 130. The destination network address (e.g., IP address) is the IP address of the GVM 106 to which it is connected. Therefore, the egress hop provides the decapsulated data message to that machine.

[0049] exist Figure 1 In some examples, the egress hop is communicatively connected to the destination machine of the data message without having to traverse an intermediate routing / switching fabric. In other examples, the egress hop is connected to the destination of the message (e.g., a destination computer, virtual machine, or container) through an intermediate fabric (e.g., through an intermediate router / switch). In these cases, the egress hop forwards the data message through the intermediate fabric.

[0050] Some embodiments use the Generic Network Virtualization Encapsulation (Geneve) tunneling protocol to carry path information (e.g., a next hop index and a list of hops along the path) and a tenant identifier. In some embodiments, the path information is embedded in a Geneve option TLV (type, length, value) that is checked by the hops along the path to identify the next hop (if any).

[0051] Figure 2 The Geneve tunnel header 200 used in some embodiments is illustrated. In some embodiments, the tunnel header 200 is a standard Geneve tunnel header with an option TLV to store path information and a next hop index. Specifically, as shown in the figure, the tunnel header includes an outer header 205, a protocol field 210, a Geneve base header 215, and an option TLV 220. As described above, the outer portion 205 of the tunnel header 200 includes a network address (e.g., the source and destination IP addresses of the two endpoints of the tunnel, where the source and destination designation depends on the direction of the message flow) that allows the encapsulated data message to pass through the underlying network and reach the tunnel destination endpoint. The protocol field 210 specifies the UDP protocol and the attributes associated with the protocol.

[0052] Figure 3 The Geneve base header 215 of some embodiments is illustrated. As shown, in some embodiments, this header is 64 bits wide and stores several tunnel parameters, including a tenant identifier and an option length. Its version field 305 is a 2-bit value that specifies the current Geneve tunnel version in use. A tunnel endpoint that receives a message with an unknown version will discard the message. Non-endpoint devices that process Geneve packets with unknown version numbers will treat these packets as UDP packets with unknown payloads.

[0053] The option length field 310 specifies the length of the option field, which is expressed in multiples of four bytes, but does not include the eight-byte fixed tunnel header. This makes the total size of the Geneve header a minimum of 8 bytes and a maximum of 260 bytes. The start of the payload header can be found using the offset from the end of the basic Geneve header. The O bit 315 specifies whether the data message is an OAM frame containing a control message instead of a data payload. The endpoint will not forward the payload of this message, and the transit device will not attempt to interpret or process it. Because control messages are infrequent, endpoints typically direct these messages to a high-priority control queue. The transit device will not change forwarding behavior, such as ECMP link selection, based on this bit.

[0054] The C bit 320 is a critical option field that indicates the presence of options when set. One or more options may set this critical bit. If this bit is set, the tunnel endpoint will parse the option list to interpret all critical options. After this bit is set, if the endpoint does not recognize the option, they must discard the encapsulated message. On devices that do not support option parsing, frames will be discarded based on the "C" bit in the base header. If this bit is not set, the tunnel endpoint can strip all options using the "Option Length (Opt Len)" and forward the decapsulated frame. Transit devices will not discard or modify packets based on this bit.

[0055] The first set of reserved bits 325 are 6 bits that must be zero when transmitting and are ignored when receiving. The protocol type bits 330 are 16 bits that indicate the type of protocol data unit that appears after the Geneve header. The VNI bits 335 represent a 24-bit identifier for a unique element of the virtual network. In many cases, this can represent an L2 segment; however, the control plane defines the forwarding semantics of the decapsulated packets. The VNI can be used as part of ECMP forwarding decisions when load balancing across CPUs, or as a mechanism to distinguish overlapping address spaces contained in encapsulated packets. The second set of reserved bits 340 are 8 bits that must be zero when sending and are ignored when receiving.

[0056] The Geneve basic header is followed by an option TLV 220. This TLV contains a four-byte option header and a variable amount of option data interpreted according to the option type. In some embodiments, the option data includes the IP addresses of the hops along the path and a next hop index that identifies one of these IP addresses as the IP address of the next hop along the path.

[0057] Figure 4A process 400 is conceptually illustrated of some embodiments performed by an ingress hop (such as hop 120). The process selects a path for a data message sent by a tenant GVM on a host computer, and then embeds a path hop list defining the path in a Geneve tunnel header that it uses to encapsulate the data message before sending the encapsulated message along the tunnel to the first hop along the selected path. In some embodiments, the ingress hop includes an SFE (e.g., a software switch) and path processing filters executed on a host computer (e.g., computer 104), and a tenant GVM (e.g., GVM 102) that sends the data message. In some embodiments, the GVM on the host computer is communicatively connected to the SFE, and the SFE forwards data messages to and from the GVM.

[0058] As shown, process 400 initially (at 405) receives a data message from a tenant GVM (e.g., GVM 102) executing on its host (e.g., 104). In some embodiments, process 400 receives the data message from a callback function defined in a port of a SFE or VNIC of the GVM. The callback function intercepts the data message sent by the GVM and forwards the data message to a path processing filter.

[0059] Based on a set of attributes associated with the data message, a process (e.g., a path processing filter) performs (at 410) a classification operation on the data message to select a defined path through the network to a destination of the data message. For at least some of the data messages, the classification operation selects a path from a set of two or more candidate paths, which are feasible candidate paths that have been specifically defined (e.g., by a controller or forwarding element) to reach the destination of the data message.

[0060] In some embodiments, the classification operation is based on a set of attributes associated with the data message. In some embodiments, the set of attributes includes only layer 2-4 header values ​​of the data message. However, in other embodiments, the set of attributes includes context attributes related to the data message, such as the traffic type of the data message (i.e., the type of content carried in the data message stream), QoS level, layer 7 parameters, process identifier, user identifier, group identifier, etc. When the set of context attributes includes the traffic type of the data message, the process selects (at 410) a path defined for the data message stream based on the traffic carried in the message stream. In some embodiments, the process 400 uses a deep packet inspector on the host computer 104 to perform deep packet inspection on the data message or its stream to identify the traffic type carried in the data message or its stream.

[0061] Based on the attributes of the data message, in some embodiments, the process selects the best path for the data message from the perspective of one or more metrics (such as cost, delay, latency, loss, etc.). In some embodiments, the process performs a classification operation (at 410) by comparing one or more attributes of the data message (e.g., a 5-tuple identifier of the data message and / or associated context attributes) with identifiers of a number of path definition rules stored in a rule storage device. Each rule specifies a path through a network, which in some embodiments is specified as the IP addresses of each hop along the path. Therefore, after matching the attributes of the data message with the set of identifiers of the rules, the process will select the path specified by the rule as the path for the data message.

[0062] It should be noted that process 400 allows two different tenants of a multi-tenant network to optimize their traffic engineering paths differently. For example, for two data message flows of two tenants carrying the same type of traffic (i.e., the same type of payload content) between the same source and destination computers (on which at least one source or destination computer of each tenant is executed), process 400 may select two different paths. This may occur because one tenant may want to use a faster, lower latency path for that type of content, while another tenant may want to use a cheaper path for that type of content. To achieve this, some embodiments define different path selection rules for different tenants, while other embodiments use a tenant identifier as one of the attributes used to perform a path selection classification operation.

[0063] After identifying (at 410) the path of the data message, the process 400 embeds (at 415) the identified path in the option TLV 220 of the Geneve tunnel header 200 that will be used to encapsulate the received data message. As described above, the option TLV stores the path according to the IP address of each hop along the path. In addition, as described above, the option TLV stores these IP addresses in reverse order, where the address of the first hop appears last in the list and the address of the last hop appears first in the list.

[0064] In the Option TLV, the process also stores (at 415) a next hop value. The process sets this value to identify the last network address in the path hop list, i.e., the address of the first hop. The next hop index value is used to identify the embedded network address of the "next hop", and in some embodiments, subsequent hops use this index to identify the next hop along the path. At 415, the process 400 also embeds the VNI of the tenant associated with the source GVM that sent the received data message into the basic header 215 of the Geneve tunnel header. The embedding of the VNI allows data messages of different tenants to be forwarded using the same set of tunnels.

[0065] Next, at 420, the process configures the rest of the Geneve tunnel header. For example, the process stores its VTEP (e.g., the VTEP associated with its SFE) and the L2 and L3 network addresses of the first hop's VTEP as source and destination network addresses in the outer portion header 205 of the tunnel header 200. At 420, the process defines the UDP protocol in the Geneve tunnel header and sets any other fields that need to be defined, based on the above discussion and the tunnel deployment configuration used.

[0066] After completing the tunnel header configuration (at 420), the process passes (at 425) the encapsulated data message along the Geneve tunnel to the first hop (e.g., hop 122) along the selected path. To send the data message, in some embodiments, the process resolves the IP address of the first hop to the tunnel connecting the ingress hop to the first hop. After 420, the process ends.

[0067] Figure 5 The process 500 performed when a hop receives a Geneve encapsulated data message is illustrated. In some embodiments, the hop executes on a host along with one or more tenant machines (e.g., GVMs). As shown, the process examines (at 505) the tunnel header and determines that the received data message is addressed to it. In some embodiments, this determination is made by extracting a next hop index from the tunnel, using the index to retrieve the network address of the next hop along the path, and then determining that the address is its own IP address.

[0068] Once process 500 determines (at 505) that the received data message is addressed to it, the process determines whether it is the last hop in the path embedded in the tunnel header. In some embodiments, this will be the case when the next hop index is its lowest value (e.g., it is 0 or 1, depending on which is defined as the lowest value), which indicates that the last hop has been reached. When process 500 determines (at 510) that it is the last hop in the embedded path, it removes the tunnel header (i.e., decapsulates the encapsulated tunnel header) and forwards the data message to the destination machine identified by the destination network address identified in the original header (e.g., header 130) of the received data message.

[0069] In some embodiments, processing 500 is performed by a path processing filter executed on the same host computer as the destination machine. In some embodiments, a port of an SFE executed on a host computer passes the encapsulated data message to the path processing filter. In these embodiments, the path processing filter passes the decapsulated data message back to the SFE port (e.g., passes a handle to the message to the portion), and the SFE port processes the message or causes the SFE to process the message so that it can be forwarded to a destination computer (e.g., a destination GVM or container) executed on the same host computer. In some embodiments, the egress hop is connected to the destination of the data message (e.g., a destination computer, virtual machine, or container) through an intermediate architecture (i.e., through an intermediate router / switch). In this case, the egress hop forwards the data message through the intermediate architecture to the destination of the data message.

[0070] When process 500 determines (at 510) that it is not the last hop in the embedded path, the process adjusts (at 520) the next hop index value in the tunnel header of the received encapsulation message to identify the next hop in the path identified by the embedded path information. As described above, in some embodiments, when the path is specified in the tunnel header by a reversed list of hop IP addresses, the process adjusts the next hop index value by decrementing the value.

[0071] Next, at 525, the process adjusts the outer header value of the tunnel header to identify its own IP address as the source IP address of the encapsulated message and the IP address of the next hop (now identified by the adjusted next hop index value) as the destination IP address of the encapsulated message. At 530, the process resolves the address of the next hop to the tunnel between the current hop and the next hop, and sends the encapsulated data message along the tunnel to the next hop. In some embodiments, before sending the encapsulated data message along the resolved tunnel, the process 500 adjusts (at 530) one or more parameters in the tunnel header of the data message (e.g., options, ID, or other attributes of the next tunnel) to reflect the new attributes and / or identity of the new tunnel along which the data message will be sent. After 530, the process ends.

[0072] Figure 6A host computer 600 is illustrated for executing the GVM, hop processing forwarding elements, and path processing filters of some embodiments in some embodiments. The host computer also executes service engines that perform context-rich, attribute-based services in a data center. The host computer 600 includes several GVMs 605, software forwarding elements 610, context engines 650, service engines 630, context-based service rule storage 640, context attribute storage 645, attribute mapping storage 623, connection state cache storage 625, MUX (multiplexer) 627, and context engine policy storage 643. The service engine 630 includes a path processor 622, a deep packet inspector 624, a load balancer 626, a firewall engine 628, and one or more other service engines (not shown).

[0073] exist Figure 6 In some embodiments, the GVM 605 is executed on the hypervisor. In some embodiments, the context engine 650, the software forwarding element 610, the service engine 630, the context-based service rule storage 640, the connection state cache storage 625, the context engine policy storage 643 and the MUX 627 operate in the kernel space of the hypervisor, while the VM 605 operates in the user space of the hypervisor. In other embodiments, one or more service engines and / or context engines are user space modules (e.g., service VMs).

[0074] In some embodiments, GVM 605 is a tenant machine that serves as a data endpoint for some or all tenant data message flows in a data center. Examples of such machines include web servers, application servers, database servers, etc. In some cases, all VMs belong to one entity, such as an enterprise operating the host. In other cases, host 600 operates in a multi-tenant environment (e.g., in a multi-tenant data center), and different VMs 605 may belong to one tenant or multiple tenants.

[0075] Each VM 605 includes a GI agent 648 that communicates with a context engine 650 to provide a set of context attributes to the engine and receives instructions and queries from the engine. The communication between the context engine 650 and the GI agent 648 is relayed through MUX 627. An example of such a multiplexer is the multiplexer used by the Endpoint Security (EPSec) platform of VMware's ESX hypervisor. In some embodiments, the attributes collected by the context engine 650 from the GI agent 648 include a rich set of parameters (e.g., layer 7 parameters, process identifiers, user identifiers, group identifiers, etc.). U.S. patent application 15 / 650,251 filed on July 14, 2017 further describes the capture and use of these context attributes by the GI agent 648. U.S. patent application 15 / 650,251 is incorporated herein by reference.

[0076] As shown, in some embodiments, each GVM 605 includes a virtual network interface card (VNIC) 655. Each VNIC is responsible for exchanging messages between its VM and the SFE 610. Each VNIC is connected to a specific port 660 of the SFE 610. The SFE 610 is also connected to the host's physical network interface card (NIC) (not shown). In some embodiments, the VNIC is a software abstraction created by the hypervisor of one or more physical NICs (PNICs) of the host.

[0077] In some embodiments, SFE 610 maintains a single port 660 for each VNIC of each VM. SFE 610 is connected to the host PNIC (through a NIC driver (not shown)) to send outgoing messages and receive incoming messages. In some embodiments, SFE 610 is defined to include port 665, which is connected to the driver of the PNIC to send messages to and receive messages from the PNIC. SFE 610 performs message processing operations to forward messages received on one of its ports to another of its ports. For example, in some embodiments, the SFE attempts to match the message to a flow-based rule using data in the message (e.g., data in the message header) and performs the action specified by the matching rule upon finding a match (e.g., passing the message to one of its ports 660 or 665, the latter indicating that the message is provided to the destination VM or PNIC).

[0078] In some embodiments, SFE 610 is a software switch, while in other embodiments, it is a software router or a combined software switch / router. In some embodiments, SFE 610 implements one or more logical forwarding elements (e.g., logical switches or logical routers) where the SFEs are executed on other hosts in a multi-host environment. In some embodiments, the logical forwarding elements can span multiple hosts to connect VMs that are executed on different hosts but belong to one logical network.

[0079] Different logical forwarding elements can be defined to specify different logical networks for different users, and each logical forwarding element can be defined by multiple software forwarding elements on multiple hosts. Each logical forwarding element isolates the traffic of a VM of one logical network from the VM of another logical network served by another logical forwarding element. Logical forwarding elements can be connected to VMs executed on the same host and / or different hosts. In some embodiments, the SFE extracts a logical network identifier (e.g., VNI) and a MAC address from a data message. The SFE in these embodiments uses the extracted VNI to identify a logical port group, and then uses the MAC address to identify a port within the port group.

[0080] Software switches (e.g., a hypervisor's software switch) are sometimes called virtual switches because they operate in software and they provide VMs with shared access to the host's PNIC(s). However, in this document, software switches are referred to as physical switches because they are items in the physical world. This term also distinguishes software switches from logical switches, which are an abstraction of the type of connectivity provided by a software switch. There are various mechanisms for creating logical switches from software switches. VXLAN provides a way to create such logical switches. The VXLAN standard is described in Mahalingam, Mallik; Dutt, Dinesh G.; et al. (2013-05-08), VXLAN: A Framework for Overlaying Virthalized Layer 6 Networks over Layer 3 Networks, IETF.

[0081] In some embodiments, the ports of SFE 610 include one or more function calls to one or more modules that implement special input / output (I / O) operations on incoming and outgoing messages received at the ports. Examples of I / O operations implemented by port 660 include ARP broadcast suppression operations and DHCP broadcast suppression operations, as described in U.S. Patent Application 9,548,965. In some embodiments of the present invention, other I / O operations (such as firewall operations, load balancing operations, network address translation operations, etc.) can be implemented in this way. By implementing a stack of such function calls, in some embodiments, the port can implement a series of I / O operations on incoming and / or outgoing messages. Moreover, in some embodiments, other modules in the data path (such as VNIC 655, port 665, etc.) implement I / O function call operations instead of port 660 or in conjunction with it.

[0082] In some embodiments, one or more function calls of the SFE port 660 may be directed to one or more service engines 630 that process service rules for data message streams sent to and received for the GVM. Some of these service rules are context-based service rules in a context-based service rule storage 640. In some embodiments, each service engine 630 has its own context-based service rule storage 640, attribute mapping storage 623, and connection state cache storage 625. Figure 6 Only one context-based service rule storage 640, attribute mapping storage 623, and connection state cache storage 625 for all service engines are presented to avoid obscuring the representation in the figure with unnecessary detail. Moreover, in some embodiments, each VM can have its own service engine instance (e.g., its own instance of path processor 622, deep packet inspector 624, load balancer 626, and firewall engine 628). In other embodiments, one service engine can provide services for data message flows of multiple VMs on a host (e.g., VMs for the same logical network).

[0083] In order to perform its service operation on the data message stream, in some embodiments, the service engine 630 attempts to match the flow identifier (e.g., a five-tuple identifier) ​​and / or the associated context attribute set of the flow to the rule identifier. Its service rules are in its context-based service rule storage device 640. Specifically, in order for the service engine 630 to perform its service inspection operation on the data message stream, the SFE port 660 of the service engine is called to supply the attribute set of the message received by the port. In some embodiments, the attribute set is a message identifier (such as a traditional five-tuple identifier). In some embodiments, one or more of the identifier values ​​can be a logical value defined for a logical network (e.g., an IP address defined in a logical address space). In other embodiments, all identifier values ​​are defined in the physical domain. In other embodiments, some of the identifier values ​​are defined in the logical domain, while other identifier values ​​are defined in the physical domain.

[0084] Then, in some embodiments, the service engine uses the attribute set of the received message (e.g., the five-tuple identifier of the message) to identify the set of context attributes that the service engine has stored for this flow in the attribute mapping storage device 623. In some embodiments, the context engine 650 supplies the context attributes for the new flow (i.e., the new network connection event) together with the flow identifier (e.g., the five-tuple identifier) ​​to the service engine 630. The context engine policy storage device 643 contains rules that control the operation of the context engine 650. In some embodiments, these policies instruct the context engine to generate rules for the service engine or instruct the service engine to generate rules. The service engines 630 in these embodiments store the context attributes they receive from the context engine in the attribute mapping storage device 623.

[0085] In some embodiments, the service engine 630 stores the set of context attributes for each new flow in an attribute map storage along with an identifier (e.g., a five-tuple identifier) ​​for the flow. In this manner, the service engine can identify the set of context attributes for each new flow it receives from the SFE port 660 by searching its attribute map storage 623 for a context record with a matching flow identifier. The context record with a matching flow identifier includes the set of context attributes for the flow.

[0086] In some embodiments, some or all of the service engines extract a set of context attributes for the new flow from the context engine. For example, in some embodiments, the service engine provides the five-tuple identifier of the new flow it receives from the SFE port 660 to the context engine 650. The engine 650 then checks its attribute storage device 645 to identify the set of attributes stored for the five-tuple identifier and then provides the set of attributes (or a subset of the set of attributes obtained by filtering the identified attribute set of the service engine) to the service engine.

[0087] Some embodiments implement an extraction model by encoding a set of attributes for a new message flow using a service token. When notified of a new network connection event, in some embodiments context engine 650 (1) collects a set of context attributes for the new event, (2) filters this set to discard attributes that are not relevant to performing one or more services on the flow, (3) stores the remaining filtered attribute subset in attribute storage 645 along with the service token, and (4) provides the service token to GI agent 648. GI agent 648 then causes this token to be transmitted to the service engine(s) in-band (e.g., in a header of a data message sent by the agent's VM to a destination) or out-of-band (i.e., separate from the data message sent from the agent's VM to a destination).

[0088] When the service engine obtains a new flow through SFE port 660, it provides the service token for this flow to the context engine, which uses this service token to identify the context attributes to be provided to the service engine in its attribute store 645. In embodiments where the SFE port does not provide this service token to the service engine, the service engine must first identify the service token by searching its data store using the identifier of the flow before supplying the service token to the context engine.

[0089] After identifying the set of contextual attributes of the data message flow, in some embodiments, the service engine 630 performs its service operations based on the service rules stored in the context-based service rule storage 640. To perform its service operations, the service engine 630 matches the received attribute subset with the corresponding attribute set stored for the service rule. In some embodiments, each service rule in the context-based service rule storage 640 has a rule identifier and a set of action parameters.

[0090] As described above, in some embodiments, the rule identifier of the service rule can be defined based on one or more context attributes that are not L2-L4 header parameters (e.g., L7 parameters, user identifiers, group identifiers, process names, loaded module identifiers, consumption parameters, etc.). In some embodiments, the rule identifier can also include L2-L4 header parameters. Moreover, in some embodiments, one or more parameters in the rule identifier can be specified based on individual values ​​or wildcard values. Moreover, in some embodiments, the rule identifier can include a set of individual values ​​or a group identifier, such as a security group identifier, a computing configuration identifier, a network configuration identifier, etc.

[0091] To match the received attribute set to a rule, the service engine compares the received attribute set to an associated identifier of the service rule stored in the context-based service rule storage 640. After identifying a matching rule, the service engine 630 performs a service operation (e.g., a path processing operation, a firewall operation, a load balancing operation, an encryption operation, other middlebox operations, etc.) based on a set of action parameters of the matching rule (e.g., based on a path hop list, allow / drop parameters, load balancing criteria, encryption parameters, etc.).

[0092] In some embodiments, the context-based service rule storage device 640 is defined in a hierarchical manner to ensure that a message rule check will match a higher priority rule before matching a lower priority rule when a subset of attributes of a message matches multiple rules. Moreover, in some embodiments, the context-based service rule storage device 640 contains a default rule that specifies the default action for any message rule check that cannot recognize any other service rule; in some embodiments, this default rule will be a match for all possible attribute subsets and ensure that the service rule engine will return actions for all received attribute subsets. In some embodiments, the default rule will not specify a service.

[0093] For example, when the messages are part of a flow associated with a communication session between two machines, multiple messages may have the same set of message identifier attributes. Thus, after matching a data message to a service rule in the context-based service rule storage 640 based on the identified set of context attributes of the message, the service engine of some embodiments stores the service rule (or a reference to the service rule) in the connection state cache storage 625 so that the service rule can be used later for subsequent data messages of the same flow.

[0094] In some embodiments, the connection state cache storage 625 stores service rules or references to service rules identified by the service engine 630 for different sets of message identifiers (e.g., different five-tuple identifiers for identifying different data message flows). In some embodiments, the connection state cache storage 625 stores each service rule or reference to a service rule with an identifier generated from a matching set of message identifiers (e.g., a five-tuple identifier for a flow).

[0095] Before checking a particular message using the context-based service rule storage 640, the service engine 630 of some embodiments checks the connection state cache storage 625 to determine whether the storage has previously identified a service rule for the flow of the message. If not, the service engine 630 identifies a set of context attributes for the message flow and then checks the context-based service rule storage 640 to find a service rule that matches the identified set of attributes of the message and / or its five-tuple identifier. When the connection state data storage has an entry for the particular message, the service engine will perform its service operation (e.g., its path selection operation) based on the action parameter set of the service rule (e.g., based on the hop list identified by the service rule in the connection cache 625).

[0096] In some embodiments, at each hop along the path, a hop path processor processes the path information embedded in the tunnel header of the data message. On the host computer, in some embodiments, software forwarding element 610 is associated with a VTEP that serves as a Geneve tunnel endpoint. In some embodiments, for an ingress hop (e.g., hop 120), the path processor performs Figure 4 The operation of process 400. For a data message sent by GVM 605 on its host computer, path processor 622 selects an engineering path for the data message through the network to its destination based on the attributes of the data message.

[0097] As described above, these attributes may include only the flow identifier (e.g., five-tuple identifier) ​​of the message in some cases, may include only the captured context metadata of the message in other cases, or may include both the flow identifier and the captured context metadata of the message. To select a path, in some embodiments, the path processor matches the flow identifier and / or context metadata attributes of the message with the rule identifier attributes of the path processor's path specifying rules stored in the rule storage device 640. In some embodiments, each path specifying rule in the storage device 640 specifies a path according to a list of IP addresses of each hop along the path.

[0098] In some embodiments, the captured data message context includes the traffic type of the data message flow (i.e., the type of content carried in the flow). In some embodiments, the DPI engine 624 identifies the traffic type when the SFE port 660 calls the DPI engine for a data message and / or other data messages in the same flow. In some embodiments, the DPI engine provides the traffic type to the context engine 650 for storage in the attribute storage device 645, so that when the processor requests this information for the data message, the context engine 650 can subsequently provide the traffic type to the path processor 622 (e.g., by providing a flow identifier attribute or service token of the message). In other embodiments, the context engine 650 (via the path processor 622 or another service engine) pushes the traffic type of the flow to the attribute storage device 623 so that the path processor can later retrieve the traffic type of the flow when processing the data messages of the flow.

[0099] When the captured context attributes include the traffic type of the data message, the ingress hop may select between two or more traffic engineering paths to the destination of the data message, provided that two or more such paths are predefined and stored in the rule storage 640. For example, as described above with reference to Figure 1 As described above, path 180 may be optimal based on delay and / or latency considerations, while path 185 may be optimal from a cost perspective. Therefore, for the first data message flow between GVMs 102 and 106 carrying video conferencing data for company executives, the path processor matches the attributes of the data message flow with the path designation rule that identifies path 180 as the path of the flow to GVM 106. On the other hand, for the two data message flows between the two GVMs carrying chat message data between two non-executives, the path processor matches the attributes of the data message flow with the path designation rule that identifies path 185 as the path of the flow to GVM 106.

[0100] After matching the attribute set of the message with one of the path-specifying rules, the path processor 622 of the ingress node embeds the path hop list of the path-specifying rule in the Geneve tunnel header along with the other tunnel attributes described above (e.g., outer header parameters, next hop index, etc.). In some embodiments, the path processor 622 causes another of the service engines 630 to perform tunnel encapsulation. After encapsulating the data message with the Geneve tunnel header 200, the path processor returns the data message to the SFE (e.g., provides a handle to the location in memory where the encapsulated data message is stored) so that the SFE can forward the data message along the Geneve tunnel to the hop associated with the first service machine in the specified path hop list.

[0101] In some embodiments, when the path processor 622 is performing a non-entry jump operation, the path processor performs Figure 5 The operations of process 500. For example, it determines that the tunnel-encapsulated data message has a path hop list that identifies the IP address of its SFE VTEP as the address of the next hop.

[0102] Once the path processor 622 determines that the received data message is addressed to it, the processor determines whether it is the last hop in the path embedded in the tunnel header. In some embodiments, this will be the case when the next hop index identifies the address of the current hop as the first address in the path hop address list (i.e., when the next hop index has its lowest value). When the path processor 622 determines that it is the last hop in the embedded path, it will remove the tunnel header (i.e., decapsulate the encapsulated tunnel header) and pass the decapsulated data message back to the SFE port (e.g., pass the handle of the message to the part), and the SFE port processes the message or causes the SFE to process the message so that it can be forwarded to the destination machine (e.g., destination GVM or container) executing on the same host computer. In some embodiments, the egress hop is connected to the destination of the data message (e.g., destination computer, virtual machine or container) through an intermediate architecture (i.e., through an intermediate router / switch). In this case, the egress hop forwards the data message to the destination of the data message through the intermediate architecture.

[0103] When the path processor is not the last hop in the embedded path, the path processor adjusts the next hop index value in the tunnel header of the received encapsulated message to identify the next hop in the path identified by the embedded path information. As described above, in some embodiments, when the path is specified in the tunnel header by a reverse list of hop IP addresses, the path processor adjusts the next hop index value by decrementing the value.

[0104] The path processor 622 adjusts the outer header value of the tunnel header to identify the IP address of its hop as the source IP address of the encapsulated message and the IP address of the next hop (now identified by the adjusted next hop index value) as the destination IP address of the encapsulated message. The path processor resolves the address of the next hop to the tunnel between the current hop and the next hop. In some embodiments, before sending the encapsulated data message along the resolved tunnel, the path processor adjusts one or more parameters in the tunnel header of the data message to reflect the attributes and / or identity of the new tunnel along which the data message will be sent. It then provides the encapsulated data message to its SFE for forwarding along the tunnel to the next hop.

[0105] Figure 7An example of how path processing filters 622 are managed in some embodiments is illustrated. The figure illustrates a plurality of hosts 600 in a data center. As shown, each host includes a number of service engines 630, a context engine 650, a path processing filter 622, a number of GVMs 605, and SFEs 610. It also illustrates a set of controllers 705 for managing the path processing filters 622 and the service engines 630, the GVMs 605, and the SFEs 610. The hosts and the controllers are communicatively connected to each other via a network 710, which may be a local area network, a wide area network, a network of networks (such as the Internet), and the like.

[0106] The controller set provides a user interface for administrators to define context-based service rules based on context attributes, and communicates with the host through the network 710 to provide these policies. In some embodiments, the context engine 650 collects context attributes, which are passed to the management server in the controller through the network 710 so that these context attributes can be used to define policies. In some embodiments, the management server interacts with the discovery engine executed on the host computer 600 in the data center to obtain and refresh the list of all processes and services running on the GVM on the host. Then, in some embodiments, the management plane provides a rule creation interface to allow administrators to create context-based service rules for the service engine and path processor 622.

[0107] Once high-level service policies are defined in the management plane, the management plane provides some or all of these policies directly to management agents (not shown) on the host 600, and / or indirectly to these agents through a set of configuration controllers. In some embodiments, the management agent publishes the received policies as rules to the context-based service rule storage 640. In some embodiments, the agent transforms the policies before publishing them to the context-based service rule storage 640. Also, in some embodiments, the context engine 650 on the host 600 parses the policies based on the collected context attributes to generate rules for the service engine.

[0108] In some embodiments, different policies are specified for different data message flows from a source GVM based on the different traffic content carried by these flows. For example, one policy may define a path hop list that uses a low latency path for a data message flow between two GVMs for a video conference involving executives of an enterprise, while another policy may define a path hop list that uses a low cost path for a data message flow from the same two GVMs related to a chat between two non-executives.

[0109] Many of the above features and applications are implemented as software processes, which are specified as a set of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more processing units (e.g., one or more processors, cores of a processor, or other processing units), they cause the (one or more) processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, RAM chips, hard drives, EPROMs, etc. Computer-readable media do not include carrier waves and electronic signals transmitted wirelessly or through wired connections.

[0110] In this specification, the term "software" refers to firmware residing in a read-only memory or an application stored in magnetic storage that can be read into a memory for processing by a processor. In addition, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while maintaining distinct software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that implement the software inventions described herein together is within the scope of the present invention. In some embodiments, when a software program is installed to operate on one or more electronic systems, the software program defines one or more specific machine implementations that perform the operations of the software program.

[0111] Figure 8 A computer system 800 is conceptually shown that implements some embodiments of the present invention. The computer system 800 can be used to implement any of the above-described hosts, controllers, and managers. Thus, it can be used to perform any of the above-described processes. The computer system includes various types of non-transitory machine-readable media and interfaces for various other types of machine-readable media. The computer system 800 includes a bus 805, (one or more) processing units 810, a system memory 825, a read-only memory 830, a permanent storage device 835, an input device 840, and an output device 845.

[0112] The bus 805 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the computer system 800. For example, the bus 805 communicatively connects the processing unit(s) 810 with the read-only memory 830, the system memory 825, and the permanent storage device 835.

[0113] From these various memory units, the processing unit(s) 810 retrieves instructions to be executed and data to be processed in order to perform the processes of the present invention. The processing unit(s) may be a single processor or a multi-core processor in different embodiments. The read-only memory (ROM) 830 stores static data and instructions required by the processing unit(s) 810 and other modules of the computer system. On the other hand, the permanent storage device 835 is a read and write memory device. This device is a non-volatile storage unit that stores instructions and data even when the computer system 800 is turned off. Some embodiments of the present invention use a mass storage device (such as a magnetic or optical disk and its corresponding disk drive) as the permanent storage device 835.

[0114] Other embodiments use removable storage devices (such as flash drives, etc.) as permanent storage devices. Like permanent storage device 835, system memory 825 is a read and write memory device. However, unlike storage device 835, system memory is a volatile read and write memory, such as random access memory. System memory stores some instructions and data that the processor needs at run time. In some embodiments, the processing of the present invention is stored in system memory 825, permanent storage device 835, and / or read-only memory 830. From these various memory units, (one or more) processing unit 810 retrieves instructions to be executed and data to be processed in order to perform the processing of some embodiments.

[0115] The bus 805 is also connected to input and output devices 840 and 845. Input devices enable a user to pass information and select commands to the computer system. Input devices 840 include an alphanumeric keyboard and a pointing device (also called a "cursor control device"). Output devices 845 display images generated by the computer system. Output devices include printers and display devices, such as cathode ray tubes (CRTs) or liquid crystal displays (LCDs). Some embodiments include devices such as touch screens that function as both input and output devices.

[0116] Finally, if Figure 8 As shown, bus 805 also couples computer system 800 to a network 865 via a network adapter (not shown). In this manner, the computer may be part of a network of computers (such as a local area network ("LAN"), a wide area network ("WAN"), or an intranet, or a network such as the Internet), and any or all components of computer system 800 may be used in conjunction with the present invention.

[0117] Some embodiments include electronic components such as microprocessors, storage devices and memories that store computer program instructions in machine-readable or computer-readable media (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, compact disk-read only (CD-ROM), compact disk-recordable (CD-R), compact disk-rewritable (CD-RW), read-only digital versatile disks (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini SD card, micro SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable A computer readable medium may store a computer program that is executable by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer code include machine code such as produced by a compiler, and files including higher-level code that is executed by a computer, electronic component, or microprocessor using an interpreter.

[0118] Although the above discussion mainly refers to a microprocessor or multi-core processor that executes software, some embodiments are performed by one or more integrated circuits, such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). In some embodiments, such an integrated circuit executes instructions stored on the circuit itself.

[0119] As used in this specification, the terms "computer", "server", "processor", and "memory" all refer to electronic or other technical devices. These terms do not include people or groups of people. For the purposes of this specification, the terms display or being displayed mean displayed on an electronic device. As used in this specification, the terms "computer-readable medium", "multiple computer-readable media" and "machine-readable medium" are completely limited to tangible, physical objects that store information in a form that is readable by a computer. These terms do not include any wireless signals, wired download signals, and any other transient or temporary signals.

[0120] Although the present invention has been described with reference to many specific details, it will be appreciated by those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit of the present invention. For example, several figures conceptually illustrate processes. The specific operations of these processes may not be performed in the exact order shown and described. The specific operations may not be performed in a series of continuous operations, and different specific operations may be performed in different embodiments. In addition, the processes may be implemented using several sub-processes, or as part of a larger macro process.

[0121] In some embodiments, the above methods are used to express path information in a single-tenant environment. Therefore, those of ordinary skill in the art will recognize that some embodiments of the present invention are equally applicable to single-tenant data centers. Conversely, in some embodiments, when an entity (e.g., a company) is a tenant in multiple different data centers of different providers, the above methods are used to carry path information across different data centers of different data center providers. In these embodiments, the tenant identifiers embedded in the tunnel header must be unique across data centers or must be translated as they traverse from one data center to another. Therefore, those of ordinary skill in the art will understand that the present invention is not limited by the above illustrative details, but is defined by the appended claims.

Claims

1. A method for specifying a path to a destination for a data message flow through a network, the method comprising: At the ingress forwarding element of the source machine connected to the data message flow: selecting a particular path from a plurality of candidate paths, the plurality of candidate paths being feasible paths through the network to a destination of the data message flow, based on a type of content carried by the data message flow; encapsulating each data message in the flow with a tunnel header, the tunnel header storing a set of hop identifiers identifying a set of hops along a network, the set of hops defining a selected path through the network; as well as The encapsulated data message of the flow is forwarded along the network, with the set of next hop identifiers being processed by forwarding elements in the network to direct the data message to a destination.

2. The method of claim 1, wherein selecting the specific path comprises: identifying the type of content carried by the data message stream; as well as A path to the destination is selected from among a plurality of paths to the destination by identifying a record using the identified content type of the data message stream, wherein the record identifies the particular path.

3. The method of claim 1, wherein the set of next hop identifiers comprises a set of forwarding element identifiers that identify a set of forwarding elements along the selected particular path to the destination.

4. The method of claim 1, wherein the network is a multi-tenant network, the method further comprising: A tenant identifier is stored in the tunnel header to identify a tenant associated with the data message flow.

5. The method of claim 4, wherein the stored tenant identifiers allow multiple different tenants to use the same set of tunnels between the set of forwarding elements through which the set of paths is defined. 6 . The method of claim 5 , wherein the tunnel header comprises a basic tunnel header for storing the tenant identifier and an option header for storing a set of hop identifiers.

7. The method of claim 1, wherein the tunnel header encapsulates each data message by being placed outside of layer 2 and layer 3 headers of the data message.

8. The method of claim 1, wherein the tunnel header is based on a tunnel protocol, the tunnel protocol allowing the tunnel header to have a variable size by having an option field with a variable size, the set of next hop identifiers being stored in the variable size option field.

9. The method of claim 1, wherein The selected path traverses the set of hops based on a particular sequence; and The outer portion of each tunnel header includes a destination network address, which is the network address of a forwarding element that is a first hop in the particular sequence.

10. The method of claim 9, wherein at each subsequent hop in the selected path that is not the last hop in the selected path, the forwarding element processes the received encapsulated data message by: identifying the next hop identifier among the set of hop identifiers to identify a next forwarding element along the selected path; Identify the tunnel to the next hop; as well as An encapsulation header of the data message is adjusted to be forwarded along the identified tunnel, the adjusting comprising adjusting an outer portion of the tunnel header to include a network address of a next hop as a destination network address.

11. The method of claim 10, wherein the tunnel header includes a next hop index value identifying a next hop in the selected path, and each subsequent hop that is not a last forwarding element uses the next hop index value to identify a next hop identifier in the selected path.

12. The method of claim 1, wherein the hop identifier is a network address in the network.

13. A machine-readable medium storing a program, which, when executed by at least one processing unit, implements the method according to any one of claims 1 to 12.

14. An electronic device comprising: a set of processing units; as well as A machine-readable medium storing a program which, when executed by at least one of the processing units, implements the method according to any one of claims 1 to 12.

15. A system for specifying a path for a data message flow through a network, comprising means for implementing the method according to any one of claims 1-12.

Images