Allocate additional bandwidth to resources in the data center through the deployment of dedicated gateways

By deploying multiple edge forwarding components in a software-defined data center and using network address prefix and traffic group policies to dynamically allocate bandwidth resources, the problem of insufficient edge network capacity of high-bandwidth flows in SDDC is solved, and flexible bandwidth allocation and resources are achieved. efficient use.

CN117178259BActive Publication Date: 2024-12-03VMWARE INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202180097278.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2021-06-11
Filing Date
2021-12-24
Publication Date
2024-12-03
Estimated Expiration
2041-12-24

AI Technical Summary

Technical Problem

Software-defined data centers (SDDCs) handle stateful streams with high bandwidth, insufficient edge network capacity, resulting in increased management overhead, and prior art is difficult to allocate dedicated bandwidth resources for large streams within the same SDDC.

Method used

By deploying multiple edge forwarding elements, bandwidth is dynamically allocated to different sets of data message streams, bandwidth resources are allocated using network address prefixes and traffic group policies, and forwarding elements are configured in the SDDC for flexible bandwidth allocation.

Benefits of technology

It effectively improves bandwidth utilization in SDDC, reduces management overhead, allows the allocation of dedicated bandwidth resources to different entities and applications in the same SDDC, and improves the flexibility and efficiency of network resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117178259B_ABST
    Figure CN117178259B_ABST
Patent Text Reader

Abstract

Some embodiments provide a policy-driven method for deploying edge forwarding elements for tenants or applications in a public or private SDDC. For example, the method of some embodiments allows an administrator to create different traffic groups for different applications and / or tenants, deploy edge forwarding elements for different traffic groups, and configure forwarding elements in the SDDC to direct data message flows of applications and / or tenants through edge forwarding elements deployed for them. The policy-driven method of some embodiments also dynamically deploys edge forwarding elements for applications and / or tenants in the SDDC after detecting the need for edge forwarding elements based on monitored traffic flow conditions.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] Software Defined Data Centers (SDDCs) are typically protected from external networks by edge routers that perform middlebox service operations such as firewalls, network address translation (NAT), etc. All external traffic is diverted through the edge gateway. The external network bandwidth in an SDDC will be determined by the minimum of the edge gateway uplink bandwidth and the host adapter network bandwidth. There are applications with flows that require large bandwidth, which consume a lot of edge network capacity. These flows are often stateful, which requires that the traffic be handled symmetrically on the same edge router. There is currently no solution to address these requirements. Because of this, customers are often asked to split their applications across multiple SDDCs so that they can get additional external network bandwidth. Each SDDC has its own management plane and this results in management overhead. There is a need to be able to assign dedicated bandwidth resources to large flows within the same SDDC. Summary of the invention

[0002] Some embodiments of the present invention provide a method for deploying an edge forwarding element in a public or private software defined data center (SDDC). For an entity (e.g., a tenant, an enterprise, a department, etc.), the method deploys a default first edge forwarding element to handle data message flows between machines of the entity in a first network of the SDDC and machines external to the first network of the SDDC (e.g., machines outside the SDDC). The method then receives a request to allocate more bandwidth to a first set of data message flows entering or leaving the first network of the SDDC.

[0003] In response, the method deploys a second edge forwarding element to process the first set of data message flows of the entity to allocate more bandwidth to the first set of data message flows while continuing to process the second set of data message flows of the entity through the default first edge node. The method of some embodiments provides a novel way to make bandwidth available as any other user-selectable resource (such as a computing machine, a service machine, a network element, etc.) in the SDDC.

[0004] In some embodiments, the method receives a request for more bandwidth by first receiving a request to create a traffic group and then receiving a list of network addresses associated with the traffic group. The list of network addresses identifies a subset of data message flows to be processed by the second edge node. In some embodiments, the network address is a network address associated with an interface of a forwarding element for connecting a machine in the first network to the first network. In some embodiments, the method receives a list of network addresses associated with the traffic group by receiving a prefix of the network address and then receiving a request to associate the prefix of the network address with the traffic group. Based on this request, the method then creates an association between the traffic group and the prefix of the received network address.

[0005] In some embodiments, the method deploys a second edge forwarding element by configuring the second edge forwarding element to forward the first set of data messages to a forwarding element in an external network and configuring the set of forwarding elements in the first network to forward the first set of data message flows from the set of machines in the first network to the second edge forwarding element. In some embodiments, the edge forwarding element is an edge router. In some of these embodiments, the method configures the second edge forwarding element by configuring the second edge forwarding element to advertise a route to the set of machines to the forwarding element in the external network.

[0006] In some embodiments, the set of forwarding elements configured in the first network includes intermediate routers. In some of these embodiments, the method configures the set of intermediate routers by providing next-hop forwarding rules to the set of intermediate routers. Alternatively or in combination, in some embodiments, the set of configured forwarding elements includes a set of intermediate switches that implement logical switches. In these embodiments, the method configures the set of intermediate switches by providing forwarding rules to the set of intermediate switches to direct the switches to forward the first set of data message flows to the second edge forwarding element through the set of tunnels, the tunnels connecting the set of intermediate routers to the second edge forwarding element.

[0007] In some embodiments, the SDDC is a public cloud data center with a second network. In these embodiments, the first network is a private network defined in the second network to implement a virtual private cloud (VPC) for entities in the public cloud data center. The first network is an isolated, dedicated physical network in some embodiments and a logical overlay network in other embodiments.

[0008] In some embodiments, the second edge forwarding element is a gateway in a public cloud data center. In some embodiments, the method deploys the second edge forwarding element by deploying the gateway and then configuring a set of forwarding elements in a second network of the public cloud data center to forward the first set of data message flows to the deployed gateway.

[0009] In some embodiments, the method deploys the first and second edge forwarding elements by deploying the first and second edge forwarding elements as separate first and second devices in the SDDC. In some embodiments, the first and second devices are different edge forwarding devices. In other embodiments, the first and second edge forwarding devices are two different machines executed on two different host computers.

[0010] After receiving a request to allocate more bandwidth to the first set of data message flows, the method of some embodiments receives a request to allocate more bandwidth to a third set of data message flows of an entity entering or leaving the first network of the SDDC. The method deploys a third edge forwarding element for the entity to process the third set of data message flows so as to allocate more bandwidth to the third set of data message flows while continuing to process the second set of data message flows by the default first edge node and the first set of data message flows by the second edge node.

[0011] As with the request to allocate more bandwidth for the first set of data message streams, in some embodiments, the method receives a request for more bandwidth for a third set of data message streams by first receiving a request to create another traffic group, receiving another prefix of a network address identifying the third set of data message streams, and then receiving a request to associate the newly received traffic group with the newly received address prefix. In some embodiments, the address prefixes for the first and third data message streams may overlap. In such cases, the method resolves the overlap by assigning the overlapping addresses to a traffic group that more specifically identifies the addresses. For example, if a first list of addresses for the first set of data message streams is specified according to a range of IP addresses (192.168.200.0 / 24), and a second list of addresses for the third set of data message streams specifies a specific address within this range (192.168.200.10), then the method assigns the more specific address to the second traffic group identifying the third set of data message streams.

[0012] In some embodiments, the method deploys the second and third edge forwarding elements by deploying them as different forwarding appliances, while in other embodiments, it deploys these forwarding elements by deploying different machines executing on different host computers in the SDDC. Using different host computers to implement different edge forwarding elements for different sets of data message flows allows dedicated resources (e.g., physical network interface cards (PNICs)) of different host computers to be used for different sets of data message flows.

[0013] Some embodiments provide a policy-driven method for deploying edge forwarding elements for tenants or applications in a public or private SDDC. For example, the method of some embodiments allows an administrator to create different traffic groups for different applications and / or tenants, deploy edge forwarding elements for different traffic groups, and configure forwarding elements in the SDDC to direct data message flows of applications and / or tenants through the edge forwarding elements deployed for them.

[0014] The policy-driven method of some embodiments also deploys edge forwarding elements for applications and / or tenants in the SDDC after detecting the need for edge forwarding elements based on monitored traffic flow conditions. For example, the method of some embodiments deploys a first edge forwarding element for a set of one or more applications to process a data message stream associated with the application set. The method detects that the data message stream associated with the application set consumes more than a threshold amount of bandwidth. Based on a policy that specifies that additional bandwidth is allocated to the data message stream when the data message stream associated with the application set consumes more than a threshold amount, it is determined in response to the detection that additional bandwidth needs to be allocated to the data message stream associated with the application set, and then, a second edge forwarding element is deployed for the application set to process at least a portion of the data message stream associated with the application set so as to allocate more bandwidth to the application set. In some embodiments, the deployment, detection, and determination operations are performed by a set of one or more controllers.

[0015] In some embodiments, the application set includes only one application implemented by a plurality of application instances executed on a plurality of host computers, all of which execute a common set of operations of the application. Before the second edge forwarding element is deployed, the first edge forwarding element processes all data message flows of all application instances of the application. After the second edge forwarding element is deployed, the first edge forwarding element processes the data message flows of the first set of application instances of the application, and the second edge forwarding element processes the data message flows of the second set of application instances of the application.

[0016] In combination or alternatively, in some embodiments, the application collection includes a first application and a second application different from the first application, the first application is implemented by several application instances executed on one or more host computers to perform a common set of operations of the first application, and the second application is implemented by several application instances executed on a second set of one or more host computers to perform a common set of operations of the second application.

[0017] Before the second edge forwarding element is deployed, the first edge forwarding element processes all data message flows of all application instances of the first application and the second application. After the second edge forwarding element is deployed, the first edge forwarding element processes the data message flows of the application instances of the first application, and the second edge forwarding element processes the data message flows of the application instances of the second application.

[0018] In some embodiments, the application set application set includes a multi-component application having multiple components executed on multiple computers. Before the second edge forwarding element is deployed, the first edge forwarding element processes all data message flows of each component of the application. After the second edge forwarding element is deployed, the first edge forwarding element processes the data message flow of the first component of the first application, and the second edge forwarding element processes the data message flow of the second component of the application.

[0019] In combination or alternatively, the method of some embodiments deploys a first edge forwarding element for a tenant in a multi-tenant SDDC to process data message flows associated with a tenant's machine operating in the SDDC. The method then detects that these data message flows consume bandwidth exceeding a threshold amount. Based on a policy that specifies that additional bandwidth is allocated to data message flows associated with a tenant when the tenant's data message flows consume more than a threshold amount, the method determines in response to the detection that additional bandwidth needs to be allocated to data message flows to and / or from the tenant's machine, and then deploys a second edge forwarding element for the tenant to process at least a portion of its data message flows so as to allocate more bandwidth to the tenant's machine.

[0020] In some embodiments, the deployment, detection, and determination operations are performed by a collection of one or more controllers. Furthermore, in some embodiments, the SDDC is a data center belonging to a multi-tenant public cloud operated by a public cloud provider that provides computing resources, network resources, and / or storage resources from multiple tenants. In other embodiments, the SDDC is a private data center of an entity (e.g., a company, school, organization, etc.), and the tenants are different sub-entities (e.g., divisions, departments, etc.) associated with the entity.

[0021] After the second edge forwarding element is deployed for the tenant, the first edge forwarding element continues to process the first set of data message flows associated with the tenant, while the second edge forwarding element processes the second set of data message flows associated with the tenant. In some embodiments, the first set of data message flows is for a first set of machines of the tenant, while the second set of data message flows is for a second set of machines of the tenant. The two sets of data message flows (i.e., the first and second data message flows) are between machines in a first network defined for the tenant in the SDDC and machines outside the first network of the SDDC (i.e., are flows entering or leaving the first network).

[0022] The foregoing summary is intended to serve as a brief introduction to some embodiments of the present invention. It is not meant to be an introduction or overview of all inventive subject matter disclosed in this document. The following detailed description and the accompanying drawings cited in the detailed description will further describe the embodiments described in the summary as well as other embodiments. Therefore, in order to understand all the embodiments described in this document, a comprehensive review of the summary, detailed description, accompanying drawings, and claims is required. Moreover, the claimed subject matter is not limited by the illustrative details in the summary, detailed description, and accompanying drawings, but is defined by the appended claims, because the claimed subject matter may be implemented in other specific forms without departing from the spirit of the subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The novel features of the invention are set forth in the appended claims.For purposes of illustration, however, several embodiments of the invention are set forth in the following drawings.

[0024] Figure 1-3 An example of deploying multiple edge gateways in an SDDC to allocate additional bandwidth to multiple different sets of inbound and outbound flows to and from machines deployed for an entity in the SDDC is illustrated.

[0025] Figure 4 Conceptually illustrates a process performed by a manager and controller server in some embodiments to define and deploy traffic groups to allocate additional bandwidth to a collection of machines.

[0026] Figure 5 An example of an administrative user interface of some embodiments for defining and creating traffic groups is illustrated.

[0027] Figure 6 Illustrate the display window that appears after selecting the flow group control.

[0028] Figure 7 Illustration of adding a newly created traffic group to the traffic groups listed in the Traffic Groups pane.

[0029] Figure 8 An IP Prefix List pane is shown including an Add IP Prefix List control.

[0030] Fig. 9 Shows selection of a prefix list control.

[0031] Fig.10 Illustrate the display window that is presented after selecting the prefix list control.

[0032] Fig.11 The Set Prefix window is shown, and Fig.12 Illustration of the Prefix pane.

[0033] Fig.13 Illustrated is a Set Prefix window displaying specified prefixes along with a user's selection of an Apply control to direct a management server to associate a specified prefix list with a prefix name.

[0034] Fig.14 Illustration of the Prefix pane after selecting the Apply control.

[0035] Figure 15-18 The association of a list of received network addresses with a traffic group is illustrated.

[0036] Fig.19 The diagram shows an example of deploying different edge gateways for different tenants and applications.

[0037] Fig. 20 The diagram illustrates an example of deploying different edge gateways for different components of a tenant's multi-component application in a multi-tenant SDDC.

[0038] Fig.21 Illustrated are policy-driven approaches of some embodiments for deploying edge forwarding elements to applications and tenants in an SDDC after detecting the need for an edge forwarding element based on monitored traffic flow conditions.

[0039] Fig. 22 Illustrated are policy-driven methods of some embodiments for deploying edge forwarding elements for application components of a multi-component application in an SDDC after detecting a need for an edge forwarding element based on monitored traffic flow conditions.

[0040] Fig.23 Conceptually illustrates the process performed by a network manager to define policies for dynamically creating edge gateways to allocate more bandwidth to specific applications or tenants.

[0041] Fig.24 Conceptually illustrates the processes performed by a set of controllers based on Fig.23 The process conceptually illustrated in FIG. 1 is used to dynamically deploy edge gateways according to the policies specified in FIG.

[0042] Fig.25 A computer system is illustrated that may be used to implement each of some embodiments of the invention. DETAILED DESCRIPTION

[0043] In the following detailed description of the present invention, many details, examples and embodiments of the present invention are set forth and described. However, it will be apparent to those skilled in the art that the present invention is not limited to the embodiments set forth, and the present invention can be practiced without discussing some of these specific details and examples.

[0044] Some embodiments of the present invention provide a method for deploying an edge forwarding element in a public or private software defined data center (SDDC). For an entity (e.g., a tenant, an enterprise, a department, etc.), the method deploys a default first edge forwarding element to handle data message flows between machines of the entity in a first network of the SDDC and machines external to the first network of the SDDC (e.g., machines outside the SDDC). The method then receives a request to allocate more bandwidth to a first set of data message flows entering or leaving the first network of the SDDC.

[0045] In response, the method deploys a second edge forwarding element to process the first set of data message flows of the entity to allocate more bandwidth to the first set of data message flows while continuing to process the second set of data message flows of the entity through the default first edge node. The method of some embodiments provides a novel way to make bandwidth available as any other user-selectable resource (such as a computing machine, a service machine, a network element, etc.) in the SDDC.

[0046] In some embodiments, the method receives a request for more bandwidth by first receiving a request to create a traffic group and then receiving a list of network addresses associated with the traffic group. The list of network addresses identifies a subset of data message flows to be processed by the second edge node. In some embodiments, the network address is a network address associated with an interface of a forwarding element for connecting a machine in a first network to the first network. In some embodiments, the method receives a list of network addresses associated with a traffic group by receiving a prefix of the network address and then receiving a request to associate the prefix of the network address with the traffic group. Based on this request, the method then creates an association between the traffic group and the prefix of the received network address.

[0047] In some embodiments, the method deploys a second edge forwarding element by configuring the second edge forwarding element to forward the first set of data messages to a forwarding element in an external network and configuring the set of forwarding elements in the first network to forward the first set of data message flows from the set of machines in the first network to the second edge forwarding element. In some embodiments, the edge forwarding element is an edge router. In some of these embodiments, the method configures the second edge forwarding element by configuring the second edge forwarding element to advertise a route to the set of machines to the forwarding element in the external network.

[0048] After receiving a request to allocate more bandwidth to the first set of data message flows, the method of some embodiments receives a request to allocate more bandwidth to a third set of data message flows of an entity entering or leaving the first network of the SDDC. The method deploys a third edge forwarding element for the entity to process the third set of data message flows so as to allocate more bandwidth to the third set of data message flows while continuing to process the second set of data message flows by the default first edge node and the first set of data message flows by the second edge node.

[0049] As with the request to allocate more bandwidth for the first set of data message flows, in some embodiments, the method receives a request for more bandwidth for a third set of data message flows by first receiving a request to create another traffic group, receiving another prefix of network addresses identifying the third set of data message flows, and then receiving a request to associate the newly received traffic group with the newly received address prefix. In some embodiments, the address prefixes for the first and third data message flows may overlap. In such cases, the method resolves the overlap by assigning the overlapping addresses to the traffic group that more specifically identifies the addresses.

[0050] For example, if a first list of addresses for a first set of data message flows is specified according to a range of IP addresses (192.168.200.0 / 24), and a second list of addresses for a third set of data message flows specifies a specific address within this range (192.168.200.10), then the method assigns the more specific address to a second flow group identifying the third set of data message flows. Alternatively, the first list of addresses may be specified according to a first range of IP addresses (192.168.200.0 / 24), and the second list of addresses may be specified as a smaller second range of IP addresses within the first range (192.168.200.0 / 32). In this case, the method assigns the more specific address (i.e., the smaller range 192.168.200.0 / 32) to the second flow group identifying the third set of data message flows and assigns the remaining IP addresses within the larger range (the remaining addresses in 192.168.200.0 / 24) to the first flow group.

[0051] In some embodiments, the method deploys the second and third edge forwarding elements by deploying them as different forwarding appliances, while in other embodiments, it deploys the forwarding elements by deploying different machines executing on different host computers in the SDDC. Using different host computers for different sets of data message flows allows dedicated resources (e.g., physical network interface cards (PNICs)) of different host computers to be used for different sets of data message flows.

[0052] As used in this document, a data message refers to a set of bits in a specific format sent across a network. One of ordinary skill in the art will recognize that the term data message is used in this document to refer to various formatted sets of bits sent across a network. The formatting of these bits can be specified by a standardized protocol or a non-standardized protocol. Examples of data messages that follow standardized protocols include Ethernet frames, IP packets, TCP segments, UDP datagrams, etc. Moreover, as used in this document, references to L2, L3, L4, and L7 layers (or Layer 2, Layer 3, Layer 4, and Layer 7) refer to the second data link layer, the third network layer, the fourth transport layer, and the seventh application layer of the OSI (Open Systems Interconnection) layer model, respectively.

[0053] In some embodiments, the edge forwarding element is an edge gateway that connects the entity's private first network to an external network (eg, a network connected to the SDDC or to an external network outside the SDDC). Figure 1-3 An example of deploying multiple edge gateways in an SDDC to allocate additional bandwidth to multiple different sets of ingress and egress flows to and from machines deployed in the SDDC for an entity is illustrated. In this example, the SDDC is a public cloud availability zone 102 in which a virtual private cloud (VPC) 100 has been defined for an entity, which in this example is a tenant of the private cloud. In some embodiments, the availability zone includes a data center or more than one data center in close proximity to each other. Although Figure 1-3 Some embodiments are illustrated for use in a public cloud context, but one of ordinary skill will recognize that some embodiments of the invention may similarly be implemented in a private data center.

[0054] For entities, VPC 100 includes a private network 105 formed by several forwarding elements (e.g., switches and routers), which are not shown in these figures to avoid cluttering these figures with unnecessary details. The forwarding elements include software forwarding elements (e.g., software switches and / or routers) and middlebox elements (e.g., firewalls, load balancers, etc.) executed on multi-tenant host computers 115 and machines 110 that have been deployed for the entity. In some embodiments, the forwarding elements also include hardware forwarding elements and / or middlebox elements (e.g., hardware switching and / or router appliances, and / or middlebox appliances).

[0055] In some embodiments, private network 105 is established by sharding the internal network address space of the private cloud and providing private network 105 with a set of internal network addresses that do not overlap with the internal network addresses provided to any other tenant of the VPC. In other embodiments, private network 105 is a logical overlay network formed by establishing tunnels between forwarding elements of the private network and having the forwarding elements exchange data messages through these tunnels, for example, by encapsulating data messages with tunnel headers (which allows data messages to be exchanged between forwarding elements) while retaining the original data message headers containing the network addresses defined in the logical address space. In some embodiments, the logical address space of one tenant can overlap with the logical address space of another tenant, but this is not important because the tunnel header is encapsulated.

[0056] Figure 1 A default gateway 120 is illustrated, which is initially deployed by a set of controllers 130 to connect the VPC network 105 with a first external network. The first external network in this example is a network inside the public cloud data center 102. In this example, any VPC gateway (including the default gateway 120) connects to (i.e., forwards packets to) one or more gateways 135 of the public cloud data center 102, which communicate with an external network 145 outside the public cloud data center 102. In other embodiments, the VPC gateways (including the default gateway 120) connect directly to the external network 145 without having to go through any gateway 135 of the public cloud data center 102.

[0057] In some embodiments, the controller set 130 configures the default gateway 120 to forward ingress data messages from the cloud gateway 135 to the VPC network, and to forward egress data messages from the VPC network to the cloud gateway 135. In some embodiments, the controller set also configures the forwarding elements in the VPC network 105 to forward egress data messages to the default gateway 120, and to forward ingress data messages to the machines 110 of the VPC network.

[0058] Figure 2 VPC 100 is illustrated after a gateway 220 has been created for a first traffic group (TG). This traffic group includes a collection 200 of machines, including machines 110d and 110e. In some embodiments, collection 200 of machines includes a group of machines for which an administrator of the entity has requested more bandwidth. In some embodiments, the administrator requests this additional bandwidth by first creating a traffic group in a management portal provided by collection 125 of manager servers and then providing a list of network addresses associated with the traffic group.

[0059] In some embodiments, the list of network addresses is network addresses associated with interfaces used to connect machines in machine set 200 to forwarding elements in VPC network 105. In some embodiments, the administrator provides a list of network addresses associated with a flow group by first providing a prefix of the network address and then requesting that this prefix of the network address be associated with the flow group. Based on this request, manager server 125 instructs controller server 130 to create an association between the flow group and the prefix of the received network address.

[0060] The list of network addresses provided by the administrator for the first TG identifies a subset of data message flows to be processed by the gateway 220 of the first traffic group. Specifically, for the first traffic group, the controller set 130 deploys the first TG gateway 220. In some embodiments, it is important that the same TG gateway handles the ingress and egress data message flows for the traffic group machines because the gateway needs to maintain state and / or perform stateful middlebox services (such as firewalls, load balancing, etc.) for the traffic group. In some embodiments, each gateway (e.g., the default gateway and each TG gateway) maintains state and / or performs stateful middlebox services on the ingress and / or egress traffic entering and / or leaving the VPC network.

[0061] In some of these embodiments, the controller set employs destination-side routing to ensure that the cloud gateway 135 forwards all ingress data messages to the first traffic group (i.e., all data messages destined for the list of network addresses provided for the first traffic group) to the TG gateway 220, and employs source-side routing to ensure that the forwarding elements of the VPC network 105 forward all egress data messages from the first traffic group (i.e., all egress data messages from the list of network addresses provided by the first traffic group) to the TG gateway 220.

[0062] More specifically, the controller set 130 configures the cloud gateway 135 to forward ingress data messages destined for the network address provided for the first traffic group to the first TG gateway 220. The controller set 130 also configures the first TG gateway 220 to forward these ingress data messages from the cloud gateway 135 to the VPC network 105, and to forward egress data messages from the first TG machine 200 to the cloud gateway 135. In some embodiments, the controller server also configures the first TG gateway 220 to advertise routes to the list of TG-associated network addresses to the cloud gateway 135. In some embodiments, the controller set 130 also configures forwarding elements in the VPC network 105 to forward egress data messages with source addresses in the list of addresses of the provided first traffic group (i.e., all egress data messages from the set 200 of machines of the first traffic group) to the first TG gateway 220. It also configures these forwarding elements to forward ingress data messages destined for the TG-associated network address to the machine set 200.

[0063] In some embodiments, the forwarding elements in the VPC network 105 include intermediary routers. In some embodiments, the controller set 130 configures these intermediary routers in the VPC network 105 by providing next-hop forwarding rules to the set of intermediary routers. Alternatively or in combination, in some embodiments, the set of configured forwarding elements includes a set of intermediary switches that implement logical switches. In these embodiments, the method configures the set of intermediary switches by providing forwarding rules to the set of intermediary switches to instruct the switches to forward the first set of data message flows to the first TG gateway 220 by connecting the set of intermediary switches to the tunnel of the first TG gateway 220.

[0064] Figure 3 The VPC 100 is illustrated after a gateway 320 has been created for a second traffic group (TG). This traffic group includes a set 300 of machines, including machines 110b and 110c. In some embodiments, the set 300 of machines includes a group of machines for which an entity administrator has requested more bandwidth. In some embodiments, the administrator requests this additional bandwidth by first creating a second traffic group in the management portal and then providing a list of network addresses associated with the second traffic group. In some embodiments, the list of addresses provided is network addresses associated with interfaces used to connect the machines in the set 300 of machines to forwarding elements in the VPC network 105. As with the addresses for the first traffic group, in some embodiments, the administrator provides a network address for the second traffic group by first providing a prefix of the network address and then requesting that this prefix of the network address be associated with the second traffic group. Based on this request, the set of managers 125 instructs the set of controllers 130 to create an association between the second traffic group and the prefix of the network address received for this group.

[0065] For the second traffic group, the controller set 130 deploys a second TG gateway 320. As for the first traffic group, the controller set employs destination-side routing to ensure that the cloud gateway 135 forwards all ingress data messages to the second traffic group (i.e., all data messages with addresses of the network provided for the second traffic group as destinations) to the second TG gateway 320, and employs source-side routing to ensure that the forwarding element of the VPC network 105 forwards all egress data messages from the second traffic group (i.e., all egress data messages from the list of network addresses provided by the second traffic group) to the second TG gateway 320.

[0066] The controller set 130 also configures the second TG gateway 320 to forward ingress data messages from the cloud gateway 135 to the VPC network 105, and forwards egress data messages from the second TG machine 300 to the cloud gateway 135. In some embodiments, the controller set 130 also configures the second TG gateway 320 to advertise routes to the network address associated with the second traffic group to the cloud gateway 135. In some embodiments, the controller set 130 also configures the forwarding element in the VPC network 105 to forward ingress data messages destined for the second TG associated network address to the machine set 300.

[0067] After the controller set 130 configures the first TG and second TG gateways 220 and 320, the first gateway 220 forwards all ingress and egress traffic for the first traffic group machines, the second gateway 320 forwards all ingress and egress traffic for the second traffic group machines, and the default gateway 120 forwards all ingress and egress traffic for entity machines that are not in the first and second traffic groups.

[0068] In some embodiments, each gateway 120, 220 or 320 is a logical gateway implemented by a high availability (HA) pair of a physical gateway, which is in a HA active-standby configuration, as further described below. Moreover, in some embodiments, each gateway is deployed as a separate appliance. In other embodiments, each gateway is deployed as a machine executed on a host computer (e.g., a multi-tenant host computer or an independent host computer). In some embodiments of these embodiments, different gateways are deployed on different host computers to maximize the throughput of each gateway. Using different host computers to implement different gateways for different traffic groups allows the dedicated resources of different host computers (e.g., physical network interface cards (PNICs)) to be used for data message flows of different traffic groups.

[0069] Figure 4The collective process 400 that is performed by the manager and controller servers 125 and 130 in some embodiments to define and deploy traffic groups to allocate additional bandwidth to machines is conceptually illustrated. This process will be described by reference to Figure 5-18 To explain, Figure 5-18 Illustrated is an administrator's interaction with an administrative user interface (UI) 500 to create and define traffic groups. In some embodiments, the administrative server 125 provides this UI and processes administrator requests made through this UI.

[0070] As shown, process 400 initiates management UI 500 to receive (at 405) an administrator's request to create a traffic group and create a traffic group (eg, create a traffic group object) in response to the request. Figure 5 An example of an administration UI 500 is illustrated. Specifically, it illustrates a traffic group pane 505 that is displayed when an administrator (i.e., user) selects a traffic group control 502 in a side panel 507 that lists network and security controls 508. The traffic pane 505 includes two tabs, a traffic group pane 504 and an IP prefix list pane 506. Figure 5 , the flow group pane 504 is shown with one previously created flow group estg1, and the user selects the add flow group control 510 via a cursor click operation, as shown.

[0071] Figure 6 The display window 600 is illustrated as being displayed after the management server 125 selects the add flow group control 510. It also illustrates that the user provides a name (estg2) for the flow group in the name field 605 and saves the newly created flow group by selecting the save control 610 via a cursor click operation. Figure 7 This newly created traffic group estg2 is illustrated as being added to the traffic groups listed on the traffic groups pane 505 .

[0072] After creating (at 405) a traffic group, process 400 receives from the user a list of network addresses that will subsequently be associated with the traffic group. In some embodiments, the user may provide a list of addresses that will later be associated with the traffic group prior to creating the traffic group. Process 400 stores the received list of network addresses as an IP prefix list.

[0073] Figure 8-14 An administrator's interaction with the management UI 500 to create and define an IP prefix list is illustrated. Figure 8 The IP prefix list pane 506 including the add IP prefix list control 800 is shown, and Fig. 9 This control 900 is shown selected by a cursor clicking operation. Fig.10The display window 1000 presented after this selection is illustrated. It also shows that in the prefix name field 1005, the user has specified a prefix name (espfxl1). It also shows the user's selection of the settings control 1010, which results in Fig.11 The Set Prefix window 1100 shown in FIG. 1 is opened.

[0074] In the Set Prefix window 1100, the user selects the Add Prefix control 1105, which instructs the UI 500 to display Fig.12 1200. In this pane 1200, the user specifies one or more IP prefixes. In this example, one IP prefix (192.168.200.0 / 24) is specified. After specifying the IP prefix, the user selects the add control 1205, which then causes the set prefix window 1100 to display the specified prefix. This display Fig.13 , together with a user selection of an apply control 1300 to instruct the management server to associate a specified prefix list with the prefix name. Fig.14 The prefix pane 506 is shown. Fig.13 After selection of the apply control 1300 in , it now displays "1" for the prefix that has been defined for the prefix name espfxl1. Fig.14 Also shown is a user selection of a save control 1400, which instructs the management server to save the specified prefix list espfxl1, which includes its name and its specified set of IP prefixes.

[0075] After receiving (at 410 ) the list of network addresses from the user, process 400 receives a request from the user to associate the received list of network addresses with the traffic group specified at 405 . Figure 15-18 An example of such an association request for some embodiments is illustrated. Fig.15 The user is shown invoking a set of controls 1500 for a specified traffic group. In some embodiments, the user invokes this set of controls 1500 through a cursor (eg, right-click) operation or keyboard operation on the traffic group name (estg2) displayed in the traffic group pane 505.

[0076] Fig.15 Also illustrated is a user selection of an edit control 1505 in control collection 1500. This selection results in Fig.1616. As shown, the mapping window has an add mapping control 1605 that allows a user to specify one or more IP prefix mappings to a traffic group (e.g., estg1). Each mapping has a name that can be entered through the name field 1610, a gateway name that can be entered through the gateway field 1620, and a mapped IP prefix that can be entered through the prefix drop-down list 1615. In order to map a traffic group to multiple IP prefixes, in some embodiments, the add mapping control 1605 must be called multiple times, once for each mapping.

[0077] Fig.16 The mapping of the traffic group estg1 to the prefix list esprfxl1 is shown. It also shows that the name of this mapping is esmap1, and the name of the gateway is compute gateway. This name indicates the machine associated with the IP prefix esprfxl1 specified in this example. Fig.17 Illustrated is the selection of the save control 1700 of the mapping window 1600 after various values ​​have been specified in the mapping window for the traffic group estg1. Fig.18 The traffic group pane 505 after this save operation is illustrated. As shown, the traffic group pane 505 displays the properties of estg1, which now includes a mapping esmap1 to the IP prefix esprfxl1.

[0078] Once the specified traffic group is associated with the list of specified network addresses, the management server 125 instructs (at 420) the controller server to deploy a gateway for the traffic group and configure the SDDC routers to forward data message traffic for the associated IP prefix of the traffic group through this gateway. In some embodiments, the controller server 130 deploys (at 425) the TG gateway as an HA pair of physical gateways, where one physical gateway serves as an active gateway and the other physical gateway serves as a standby gateway. In some embodiments, each physical gateway is deployed as a machine (e.g., a virtual machine) executing on a host computer in the SDDC, and for HA purposes, the gateways in the active / standby pair are deployed on different host computers.

[0079] After the TG gateway is deployed, the controller server 130 configures (at 430) a cloud gateway (e.g., gateway 135) to direct all ingress data messages to the entity's VPC with a destination of the list of IP addresses of the received traffic group (e.g., the IP prefix of the TG) to the TG gateway deployed at 425. As mentioned above, the controller server configures the cloud gateway by providing a next-hop forwarding rule that identifies the TG gateway as the next hop for ingress data messages with a destination IP address in the IP prefix.

[0080] Next, at 435, the controller server 130 configures the VPC-implementing router to direct all egress data messages leaving the entity's VPC from a source having a list of IP addresses of the received traffic group (e.g., from the TG's IP prefix) to the TG Gateway deployed at 425. As mentioned above, the controller server configures the VPC-implementing router by providing next-hop forwarding rules that identify the TG Gateway as the next hop for ingress data messages having a source IP address in the IP prefix. After 435, the process ends.

[0081] Some embodiments provide a policy-driven method for deploying edge forwarding elements for tenants or applications in a public or private SDDC. For example, the method of some embodiments allows an administrator to create different traffic groups for different applications and / or tenants, deploy edge forwarding elements for different traffic groups, and configure forwarding elements in the SDDC to direct data message flows of applications and / or tenants through the edge forwarding elements deployed for them.

[0082] Fig.19 An example of such an approach is illustrated. This figure illustrates a multi-tenant SDDC 1900 that includes a number of host computers 1905 on which a number of machines 1910 (e.g., VMs, Pods, containers, etc.) and a number of edge gateways 1915a-c (e.g., edge routers) execute for a number of tenants of the SDDC 1900. In some embodiments, the edge gateways 1915a-c operate on host computers and / or standalone devices on which the machines 1910 do not execute. In some embodiments, the SDDC 1900 is a data center of a public cloud provider, while in other embodiments, the SDDC 1900 is a private data center of an entity (e.g., a company or other business entity, a school, an organization, etc.).

[0083] Fig.19 1915a is deployed to serve as a gateway for the first tenant. In this capacity, the first gateway 1915a forwards data message flows to and from the following machines: (1) machines 1910 deployed in the first VPC defined in the SDDC 1900 for the first tenant, and (2) machines outside this VPC (e.g., computers external to the SDDC 1900, or computers within the SDDC 1900 but belonging to the VPC of another tenant).

[0084] The second edge gateway 1915b is deployed to serve as a gateway for a specific application for the second tenant. In some embodiments, this application is implemented by multiple application instances that execute on multiple machines 1910 and perform a common set of operations for the application. In other embodiments, this application is a multi-component application (e.g., a multi-tier application (such as a three-tier application) or a microservice application) with multiple sets of application instances executed on multiple machines, different sets of application instances performing different operations of different components of the multi-component application, and application instances in the same set of application instances perform a common set of operations associated with their corresponding application components.

[0085] In its gateway capacity for a specific application of the second tenant, the second gateway 1915b forwards data message flows to and from: (1) machines 1910 deployed in a second VPC defined in the SDDC 1900 for the second tenant, and (2) machines outside the second VPC (e.g., machines external to the SDDC 1900, or machines within the SDDC 1900 but belonging to another tenant's VPC).

[0086] exist Fig.19 In the example of , the third gateway 1915c forwards ingress and egress data message flows to and from all other machines 1910 deployed in their respective networks in the SDDC, to and from the external network. These machines 1910 include machines of tenants 3 to N, where N is an integer. These machines 1910 also include machines of the second tenant that do not execute the specific application for which the second edge gateway 1915b is deployed.

[0087] Fig. 20 Another example of deploying different edge forwarding elements for different sets of machines is illustrated. In this example, different edge gateways are deployed for different components of a multi-component application of a tenant in a multi-tenant SDDC 2000. This figure illustrates a multi-tenant SDDC 2000 that includes a number of host computers 2005, a number of machines 2010 (e.g., VMs, Pods, containers, etc.), and a number of edge gateways 2015a-c (e.g., edge routers) executing thereon for several tenants of the SDDC 2000. In some embodiments, the edge gateways 2015a-c operate on host computers and / or standalone appliances on which the machines 2010 do not execute. In some embodiments, the SDDC 2000 is a data center of a public cloud provider, while in other embodiments, the SDDC 2000 is a private data center of an entity (e.g., a company or other business entity, a school, an organization, etc.).

[0088] Fig. 202000. A first edge gateway 2015a is deployed to serve as a gateway for component Y of a multi-component application X that is deployed for a first tenant and executed on multiple computers 2005 in SDDC 2000. In this capacity, first gateway 2015a forwards data message flows to and from: (1) machines 2010 that execute application component Y and are deployed in a first VPC defined in SDDC 2000 for the first tenant, and (2) machines outside of this VPC (e.g., computers outside the SDDC, or computers inside the SDDC but belonging to another tenant's VPC).

[0089] In some embodiments, the multi-component application X is a multi-tier application, such as a three-tier application, in which the first tier is formed by a set of one or more web servers, the second tier is formed by a set of one or more application servers, and the third tier is composed of a set of one or more database servers. In other embodiments, the multi-component application X is a microservice application with many different tier components, and each tier has many different tier application instances. In some of these embodiments, each application-component tier includes several application instances that perform a common set of operations for the application component and are executed on several machines 2010. In these embodiments, different sets of application instances deployed for different application-component tiers perform different operations for different components of the multi-component application. Fig. 20 , machine group 2050 performs operations of application component X and utilizes first edge gateway 2015a to exchange packets with machines outside the VPC defined for the first tenant in SDDC 2000.

[0090] The second edge gateway 2015b is deployed to serve as a gateway for the remaining components of the multi-component application X. In this capacity, the second gateway 2015b forwards data message flows to and from: (1) the machine 2055 on which these other components of the multi-component application X execute, and (2) machines in the SDDC 2000 that are outside the VPC deployed for the first tenant.

[0091] exist Fig. 20 In the example, the third gateway 2015c forwards ingress and egress data message flows to and from all other machines 2010 deployed in their respective networks in the SDDC 2000, to and from external networks. These machines include machines of the first tenant that are not executing the multi-component application X, as well as machines of several other tenants of the SDDC 2000. Fig. 20 , these other machines are depicted as machine group 2060.

[0092] The policy-driven method of some embodiments also deploys edge forwarding elements for applications and / or tenants in the SDDC after detecting the need for edge forwarding elements based on monitored traffic flow conditions. For example, the method of some embodiments deploys a first edge forwarding element for a set of one or more applications to process a data message stream associated with the application set. The method detects that the data message stream associated with the application set consumes more than a threshold amount of bandwidth. Based on a policy that specifies that additional bandwidth is allocated to the data message stream when the data message stream associated with the application set consumes more than a threshold amount, it is determined in response to the detection that additional bandwidth needs to be allocated to the data message stream associated with the application set, and then, a second edge forwarding element is deployed for the application set to process at least a portion of the data message stream associated with the application set so as to allocate more bandwidth to the application set. In some embodiments, the deployment, detection, and determination operations are performed by a set of one or more controllers.

[0093] In some embodiments, the application set includes only one application implemented by a plurality of application instances executed on a plurality of host computers, all of which execute a common set of operations of the application. Before the second edge forwarding element is deployed, the first edge forwarding element processes all data message flows of all application instances of the application. After the second edge forwarding element is deployed, the first edge forwarding element processes the data message flows of the first set of application instances of the application, and the second edge forwarding element processes the data message flows of the second set of application instances of the application.

[0094] In combination or alternatively, in some embodiments, the application set includes a first application and a second application different from the first application. The first application is implemented by a plurality of application instances executed on a first set of one or more host computers to perform a common set of operations of the first application, and the second application is implemented by a plurality of application instances executed on a second set of one or more host computers to perform a common set of operations of the second application.

[0095] Before the second edge forwarding element is deployed, the first edge forwarding element processes all data message flows of all application instances of the first and second applications. After the second edge forwarding element is deployed, the first edge forwarding element processes the data message flows of the application instances of the first application, while the second edge forwarding element processes the data message flows of the application instances of the second application.

[0096] In some embodiments, the application set includes a multi-component application having a plurality of components executed on a plurality of computers. Before the second edge forwarding element is deployed, the first edge forwarding element processes all data message flows for each component of the application. After the second edge forwarding element is deployed, the first edge forwarding element processes the data message flows for the first component of the first application, and the second edge forwarding element processes the data message flows for the second component of the application.

[0097] In combination or alternatively, the method of some embodiments deploys a first edge forwarding element for a tenant in a multi-tenant SDDC to process data message flows associated with a tenant's machine operating in the SDDC. The method then detects that these data message flows consume bandwidth exceeding a threshold amount. Based on a policy that specifies that additional bandwidth is allocated to data message flows associated with the tenant when the data message flows associated with the tenant consume more than a threshold amount, the method determines in response to the detection that additional bandwidth needs to be allocated to data message flows to and / or from the tenant's machine, and then deploys a second edge forwarding element for the tenant to process at least a portion of its data message flows so as to allocate more bandwidth to the tenant's machine.

[0098] In some embodiments, the deployment, detection, and determination operations are performed by a collection of one or more controllers. Furthermore, in some embodiments, the SDDC is a data center belonging to a multi-tenant public cloud operated by a public cloud provider that provides computing resources, network resources, and / or storage resources from multiple tenants. In other embodiments, the SDDC is a private data center of an entity (e.g., a company, school, organization, etc.), and the tenants are different sub-entities (e.g., divisions, departments, etc.) associated with the entity.

[0099] After the second edge forwarding element is deployed for the tenant, the first edge forwarding element continues to process the first set of data message flows associated with the tenant, while the second edge forwarding element processes the second set of data message flows associated with the tenant. In some embodiments, the first set of data message flows is for a first set of machines of the tenant, while the second set of data message flows is for a second set of machines of the tenant. The two sets of data message flows (i.e., the first and second data message flows) are between machines in a first network defined in the SDDC for the tenant and machines outside the first network of the SDDC (i.e., are flows entering or leaving the first network).

[0100] Fig.21 An example of a policy-driven approach of some embodiments is illustrated for deploying edge forwarding elements for applications and tenants in an SDDC after detecting the need for edge forwarding elements based on monitored traffic flow conditions. Fig.19 The example described shows that before the first edge forwarding element 1915a and the second edge forwarding element 1915b are deployed for the machines of tenant 1 and tenant 2 respectively executing application N, the third edge forwarding element 1915c is deployed at a first time instance to handle all external data message traffic for these machines.

[0101] The figure also illustrates that, by a second time instance, the first edge forwarding element 1915a has been deployed for machines of Tenant 1, and the second edge forwarding element 1915b has been deployed for machines of Tenant 2 executing application N. In some embodiments, the SDDC controller set deploys the first edge forwarding element 1915a and the second edge forwarding element 1915b after detecting a particular level of data message flow to the machines and / or detecting a particular level of congestion (e.g., a particular level of data messages flowing therethrough) of the third edge gateway 1915c.

[0102] Fig. 22 An example of a policy-driven approach of some embodiments is illustrated, which deploys edge forwarding elements for application components of a multi-component application in an SDDC after detecting the need for an edge forwarding element based on monitored traffic flow conditions. This figure elaborates on the policy-driven approach of some embodiments. Fig. 20 An example of description.

[0103] This diagram shows that prior to deploying a first edge forwarding element 2015a for the machines of tenant 1 executing component Y of multi-component application X, a second edge forwarding element 2015b is deployed at a first instance in time to handle all external data message traffic for all machines executing application X. It also illustrates that a third edge forwarding element 2015c is deployed at the first instance in time to handle all other external data message traffic for the first tenant and several other tenants of the SDDC.

[0104] Fig. 22 It is illustrated that, by the second time instance, the first edge forwarding element 2015a has been deployed for the Tenant 1 machine executing component Y of the multi-component application X. At this point, the second edge forwarding element 2015b handles all external data message traffic for all machines executing other components of application X (i.e., application X machines that are not executing component Y). At this point, the third edge forwarding element 2015c continues to handle all other external data message traffic for the first tenant and several other tenants of the SDDC. In some embodiments, the SDDC controller set deploys the first edge forwarding element 2015a and the second edge forwarding element 2015b after detecting a particular level of data message flow for these machines and / or detecting a particular level of congestion (e.g., a particular level of data messages flowing through) of the second edge gateway 2015b.

[0105] Fig.23The process 2300 of defining a policy for dynamically creating an edge gateway to allocate more bandwidth to a specific application or tenant is illustrated. This process 2300 is performed by a network manager, which receives user input through an API or a user interface of the network manager. As shown, when the network manager receives (at 2305) a request to define a policy for dynamically creating an edge gateway, the process 2300 begins. In some embodiments, this request is received when an administrator selects a web page through which such a policy can be created.

[0106] This page is similar to the one referenced above. Figure 5-18 Using this web page, the administrator defines the traffic groups for which the newly created edge gateways will be created, except that the web page has one or more controls as follows, through which the administrator can specify the conditions under which the traffic groups (i.e., their edge gateways) should be dynamically deployed. Other embodiments use a web page with the following controls, which are used to specify (1) the conditions under which the edge gateways should be dynamically deployed and (2) the applications, application-components, and / or tenants for which the edge gateways should be dynamically deployed.

[0107] Through a web interface or an API interface, process 2300 receives (at 2310) the identity of the application, application-component, and / or tenant for which the edge gateway should be dynamically deployed. Figure 19-22 As described, some embodiments allow for dynamically deploying edge gateways for tenants, applications, or components of multi-component applications. Once deployed, in some embodiments, this gateway will handle north / south data message flows for all machines of the tenant, machines executing all instances of the application, or machines executing all instances of the application components.

[0108] In some embodiments, the identity of an application or application component is specified based on a virtual IP address associated with the application or application component, while in other embodiments, this identity is provided by other means, such as by another identifier (e.g., a name) associated with the application or application component. Similarly, some embodiments allow an administrator to identify a tenant for which a gateway is to be dynamically deployed by a tenant identifier (e.g., an alphanumeric name associated with the tenant) or by a range of network addresses (e.g., a subnet or IP range) associated with the tenant.

[0109] Next, at 2315, process 2300 receives input about the condition that the edge gateway should be dynamically deployed. This condition may be specified differently in different embodiments. Some embodiments allow this condition to be specified based on one or more metrics (e.g., the number of packets, the number of bytes, the number of connections, the number of connections per second) associated with the gateway used by the tenant, application, or application-component before the operation of the gateway dynamically deployed for the tenant, application, or application-component. For one or more such metrics, in some embodiments, the network administrator can specify one or more thresholds through the web interface of the network manager. When the network controller set detects that any one of these thresholds or a combination of these values ​​is crossed (e.g., exceeded) based on the statistical information collected by it, the network controller set dynamically deploys the gateway for the tenant, application, or application-component.

[0110] At 2320, the process 2300 creates a policy based on the inputs provided at 2310 and 2315. This policy specifies a set of conditions under which the controller set should dynamically deploy edge gateways for tenants, applications, or application-components. Next, at 2325, the process 2300 distributes this generated policy to the controller set for implementation. After 2325, the process 2300 ends.

[0111] Fig.24 Illustrated is a process 2400 performed by a controller set to dynamically deploy edge gateways based on policies specified by process 2300. This process 2400 continuously collects (at 2405) statistical information about data message flows traversing edge gateways from forwarding elements in the SDDC that are deployed in the SDDC for tenants and / or applications for which policies for dynamically deploying edge gateways are defined.

[0112] For each policy, the process collects statistical information related to evaluating the following conditions, which are specified for the tenant, application, or application component specified by the policy to dynamically deploy the edge gateway specified by the policy. Specifically, in some embodiments, the statistical information collected for the policy is for one or more metrics used to define the conditions specified by the policy. These metrics are associated with data message streams processed by one or more other edge gateways for the set of machines for which the policy is specified (e.g., a set of machines for the tenant for which the policy is defined, or a set of machines executing applications or application components). In some embodiments, the collected statistical information is also for these metrics because they involve data message streams processed by these other gateways for other sets of machines (e.g., machines for other tenants or machines executing other applications or application instances).

[0113] In some embodiments, the process 2400 uses a pull model to actively retrieve statistics from SDDC forwarding elements, while in other embodiments, the process 2400 uses a push model to passively receive statistics from these forwarding elements. In some embodiments, the process 2400 retrieves or receives statistics only from SDDC edge gateways, while in other embodiments, the process 2400 retrieves or receives statistics from both edge and non-edge forwarding elements in the SDDC.

[0114] At 2410, process 2400 repeatedly aggregates and analyzes (at 2410) the statistical information it collects to determine whether any thresholds for deploying one or more dynamic gateways have been exceeded. In some embodiments, each threshold involves a value collected and aggregated for one metric, while in other embodiments, each threshold involves a collection of values ​​collected for a collection of metrics, which are then aggregated (e.g., by weighting and adding) through a blending function. In still other embodiments, some thresholds are associated with one metric value, while other thresholds are associated with several metric values.

[0115] At 2415, the process determines whether any thresholds have been passed. If not, then it returns to 2405 to collect additional statistics. On the other hand, when the process 2400 determines that one or more thresholds have been passed (e.g., exceeded), the process 2400 deploys (at 2420) a gateway for each policy that the process 2400 identifies as having a threshold that was triggered (i.e., passed) at 2415.

[0116] In some embodiments, deploying each edge gateway only requires creating a record in a data repository (e.g., a database table) associating a previously deployed edge gateway with a specific tenant, application, or application component. In other embodiments, this deployment requires a newly instantiated edge forwarding element to operate on a host computer or edge device.

[0117] After 2420, process 2400 configures each newly deployed edge gateway to handle northbound and southbound data message traffic from the tenant, application, or application component for which the edge gateway is deployed. In some embodiments, this configuration requires configuring the edge gateway to externally advertise routes to the SDDC network of the tenant, application, or application component for which the gateway is deployed (i.e., advertise routes to external routers). This configuration also requires configuring the edge gateway to perform destination-side routing to ensure that the SDDC gateway (e.g., cloud gateway 135) forwards all ingress data messages to the machines of the tenant, application, or application component to the newly deployed gateway.

[0118] At 2425, the process 2400 configures the intermediary architecture (i.e., intermediary forwarding elements) in the SDDC to use the newly deployed edge gateway for northbound data message traffic (i.e., for egress data message flows from the machines of the tenant, application, or application component that leave its VPC through the newly deployed gateway). In some embodiments, this configuration requires configuring these intermediary forwarding elements to perform source-side routing to ensure that the forwarding elements of the VPC network forward all egress data messages from the machines of the tenant, application, or application component to the newly deployed gateway. Source-side and destination-side routing operate in the same manner as described above.

[0119] Fig.25 A computer system 2500 for implementing some embodiments of the present invention is conceptually illustrated. The computer system 2500 can be used to implement any of the above-described hosts, controllers, and managers. Thus, it can be used to perform any of the above-described processes. This computer system 2500 includes various types of non-transitory machine-readable media and interfaces for various other types of machine-readable media. The computer system 2500 includes a bus 2505, (one or more) processing units 2510, a system memory 2525, a read-only memory 2530, a permanent storage device 2535, an input device 2540, and an output device 2545.

[0120] Bus 2505 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of computer system 2500. For example, bus 2505 communicatively connects processing unit(s) 2510 with read-only memory 2530, system memory 2525, and permanent storage device 2535.

[0121] From these various memory units, (one or more) processing unit 2510 retrieves instructions to be executed and data to be processed in order to perform the process of the present invention. In different embodiments, (one or more) processing unit can be a single processor or a multi-core processor. Read-only memory (ROM) 2530 stores static data and instructions required by (one or more) processing unit 2510 and other modules of computer system 2500. On the other hand, permanent storage device 2535 is a read-write memory device. This device 2535 is a non-volatile memory unit that stores instructions and data even when computer system 2500 is turned off. Some embodiments of the present invention use a large-capacity storage device (such as a magnetic disk or optical disk and its corresponding disk drive) as a permanent storage device 2535.

[0122] Other embodiments use a removable storage device (such as a floppy disk, flash drive, etc.) as the permanent storage device 2535. Similar to the permanent storage device 2535, the system memory 2525 is a read-write storage device. However, unlike the storage device 2535, the system memory 2525 is a volatile read-write memory, such as a random access memory. The system memory 2525 stores some of the instructions and data that the processor needs at run time. In some embodiments, the processes of the present invention are stored in the system memory 2525, the permanent storage device 2535, and / or the read-only memory 2530. The (one or more) processing unit 2510 retrieves instructions to be executed and data to be processed from these various memory units in order to perform the processes of some embodiments.

[0123] The bus 2505 is also connected to input and output devices 2540 and 2545. The input device 2540 enables a user to transmit information and selection requests to the computer system 2500. The input device 2540 includes an alphanumeric keyboard and a pointing device (also called a "cursor control device"). The output device 2545 displays images generated by the computer system 2500. The output device 2545 includes a printer and a display device, such as a cathode ray tube (CRT) or a liquid crystal display (LCD). Some embodiments include a device such as a touch screen, which acts as both an input device 2540 and an output device 2545.

[0124] Finally, if Fig.25 As shown in , bus 2505 also couples computer system 2500 to a network 2565 via a network adapter (not shown). In this manner, computer 2500 may be part of a network of computers, such as a local area network ("LAN"), a wide area network ("WAN"), or an intranet, or a network of networks, such as the Internet. Any or all of the components of computer system 2500 may be used in conjunction with the present invention.

[0125] Many of the above features and applications are implemented as software processing designated as a set of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more processing units (e.g., one or more processors, cores of a processor, or other processing units), they cause the (one or more) processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, RAM chips, hard drives, EPROMs, etc. Computer-readable media do not include carrier waves and electronic signals transmitted wirelessly or through wired connections.

[0126] In this specification, the term "software" is intended to include firmware residing in a read-only memory or an application stored in a magnetic storage device, which can be read into a memory for processing by a processor. Moreover, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while retaining different software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that implement the software inventions described herein together is within the scope of the present invention. In some embodiments, the software program defines one or more specific machine implementations of the operation of the software program when it is installed to operate on one or more electronic systems.

[0127] Some embodiments include electronic components, such as a microprocessor, that store computer program instructions in a machine-readable or computer-readable medium (alternatively referred to as a computer-readable storage medium, a machine-readable medium, or a machine-readable storage medium). Some examples of such computer-readable media include RAM, ROM, compact disk-read only (CD-ROM), compact disk-recordable (CD-R), compact disk-rewritable (CD-RW), read-only digital versatile disk (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini-SD card, micro-SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable A computer readable medium may store a computer program that is executable by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer code include machine code (e.g., produced by a compiler), and files including high-level code that is executed by a computer, electronic component, or microprocessor using an interpreter.

[0128] Although the above discussion refers primarily to microprocessors or multi-core processors that execute software, some embodiments are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves.

[0129] As used in this specification, the terms "computer", "server", "processor" and "memory" refer to electronic or other technical devices. These terms exclude a person or group of people. For the purposes of this specification, the term "display" refers to displaying on an electronic device. As used in this specification, the terms "computer-readable medium", "computer-readable medium" and "machine-readable medium" are entirely limited to tangible physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, and any other short-lived or transient signals.

[0130] Although the present invention has been described with reference to many specific details, those skilled in the art will recognize that the present invention may be implemented in other specific forms without departing from the spirit of the present invention. For example, several of the above-described embodiments allocate more bandwidth to a collection of data message flows by having an administrator request the creation of a new traffic group, associating a collection of network addresses with this traffic group, and then deploying a new gateway for the traffic group to handle the ingress / egress traffic associated with the collection of network addresses. However, other embodiments allow the administrator to simply request a specific amount (e.g., a specific amount of bytes / second) or a general amount (e.g., high, medium, low, etc.) of ingress / egress bandwidth for a collection of data message flows. Therefore, those skilled in the art will understand that the present invention is not limited by the foregoing illustrative details, but is defined by the appended claims.

Claims

1. A method for deploying an edge forwarding element in a multi-tenant software defined data center (SDDC), the method comprising: deploying a first edge forwarding element for a tenant to process a data message flow associated with the tenant; detecting that a data message flow associated with the tenant consumes bandwidth exceeding a threshold amount; determining, in response to the detecting, that additional bandwidth needs to be allocated for a data message flow associated with the tenant based on a policy that specifies allocating additional bandwidth for a data message flow when the data message flow associated with the tenant consumes more than a threshold amount, wherein the additional bandwidth is for a data message flow between a set of first machines in a first network specified by the policy and a set of second machines in a second network external to the first network, and the data message flow is a data message flow entering or exiting the first network; as well as A second edge forwarding element is deployed for the tenant to process at least a portion of the data message flow corresponding to the set of the first machines specified by the policy so as to allocate more bandwidth to the tenant, wherein deploying the second edge forwarding element includes (i) configuring the second edge forwarding element to forward a first subset of the portion of the data message flow to a forwarding element in the external second network and (ii) configuring the set of forwarding elements in the first network to forward the first subset of the data message flow from the set of the first machines in the first network to the second edge forwarding element.

2. The method of claim 1, wherein the first edge forwarding element continues to process a first set of data message flows associated with the tenant, while the second edge forwarding element processes a second set of data message flows associated with the tenant.

3. The method of claim 2, wherein the first set and the second set of data message flows are directed to two different machines of a tenant executing in the SDDC.

4. The method of claim 2, wherein the first set and the second set of data message flows are for two different instances of an application executing for a tenant in the SDDC.

5. The method of claim 2, wherein the first set and the second set of data message flows are for two different components of a multi-component application executing for a tenant in the SDDC.

6. The method of claim 2, wherein the deploying, detecting, and determining operations are performed by a collection of one or more controllers.

7. The method of claim 1 , wherein configuring the set of forwarding elements in the first network comprises configuring the set of forwarding elements to forward, to a second edge forwarding element, a data message flow having (i) a destination IP address associated with the second edge forwarding element and (ii) a source IP address associated with the set of first machines to which the additional bandwidth is allocated.

8. The method of claim 7, further comprising configuring a gateway of the SDDC to forward a data message flow having a destination IP address associated with the first set of machines to a second edge forwarding element.

9. The method of claim 1, wherein: The edge forwarding elements are edge routers, and Configuring the second edge forwarding element includes configuring the second edge forwarding element to advertise routes to the set of first machines to forwarding elements in the external second network.

10. The method of claim 1, wherein: The set of forwarding elements includes the set of intermediate routers, and Configuring the set of forwarding elements includes providing next-hop forwarding rules to the set of intervening routers.

11. The method of claim 1, wherein: The set of forwarding elements includes a set of intervening switches that implement the logical switch; and Configuring the set of forwarding elements includes providing forwarding rules to the set of intervening switches to direct the switches to forward the first set of data messages to the second edge forwarding element through the set of tunnels connecting the set of intervening switches to the second edge forwarding element.

12. The method of claim 1, wherein: SDDC is a public cloud data center with the external second network. The first network is a private network defined in the second network to implement a virtual private cloud VPC for a tenant in a public cloud data center, and Deploying the second edge forwarding element includes: deploying a gateway in a public cloud data center to serve as a second edge forwarding element for the VPC; and The set of forwarding elements in the second network is configured to forward a second subset of the portion of the data message flow from outside the VPC to the deployed gateway.

13. The method of claim 1, wherein deploying the first edge forwarding element and the second edge forwarding element comprises deploying the first edge forwarding element and the second edge forwarding element on different devices in the SDDC.

14. The method of claim 13, wherein the different devices are first and second host computers on which the first edge forwarding element and the second edge forwarding element execute.

15. The method of claim 1, wherein the SDDC comprises a public cloud data center that provides computing resources to multiple tenants.

16. The method of claim 1, wherein the SDDC is a private data center of an entity and the tenants are different sub-entities associated with the entity.

17. A method for deploying an edge forwarding element in a software defined data center (SDDC), the method comprising: For a set of one or more applications, deploying a first edge forwarding element to process a data message flow associated with the set of applications, wherein the set of applications includes a multi-component application having a plurality of components executing on a plurality of computers; detecting that a data message flow associated with the set of applications consumes bandwidth exceeding a threshold amount; determining, in response to the detecting, that additional bandwidth needs to be allocated for the data message flows associated with the set of applications based on a policy that specifies allocating additional bandwidth for the data message flows when the data message flows associated with the set of applications consume more than a threshold amount; as well as deploying a second edge forwarding element for the application set to process at least a portion of the data message flow associated with the application set so as to allocate more bandwidth to the application set, Before the second edge forwarding element is deployed, the first edge forwarding element processes all data message flows of each component of the application, After the second edge forwarding element is deployed, the first edge forwarding element processes the data message flow of the first component of the application, and the second edge forwarding element processes the data message flow of the second component of the application.

18. The method of claim 17, wherein deploying the first edge forwarding element and the second edge forwarding element comprises deploying the first edge forwarding element and the second edge forwarding element on different devices in the SDDC.

19. The method of claim 18, wherein the different devices are first and second host computers on which the first edge forwarding element and the second edge forwarding element execute.

20. The method of claim 17, wherein: The deployment, detection and determination operations are performed by a set of one or more controllers.

21. A method for deploying an edge forwarding element in a software defined data center (SDDC), the method comprising: For a set of one or more applications, deploying a first edge forwarding element to process data message flows associated with the set of applications, wherein the set of applications includes a first application and a second application different from the first application, the first application being implemented by a plurality of application instances running on a first plurality of host computers to perform a common set of operations of the first application, and the second application being implemented by a plurality of application instances running on a second plurality of host computers to perform a common set of operations of the second application; detecting that the data message flow associated with the set of applications consumes bandwidth exceeding a threshold amount; determining, in response to the detecting, that additional bandwidth needs to be allocated for the data message flows associated with the set of applications based on a policy that specifies allocating additional bandwidth for the data message flows when the data message flows associated with the set of applications consume more than the threshold amount; deploying a second edge forwarding element for the application set to process at least a portion of the data message flow associated with the application set so as to allocate more bandwidth to the application set, Before the second edge forwarding element is deployed, the first edge forwarding element processes all data message flows of all application instances of the first application and the second application, After the second edge forwarding element is deployed, the first edge forwarding element processes the data message flow of the application instance of the first application, and the second edge forwarding element processes the data message flow of the application instance of the second application.

22. A method for deploying an edge forwarding element in a software defined data center (SDDC), the method comprising: For a set of one or more applications, deploying a first edge forwarding element to process data message flows associated with the set of applications, wherein the set of applications includes only one application implemented by a plurality of application instances executing on a plurality of host computers, all of the application instances performing a common set of operations of the application; detecting that the data message flow associated with the set of applications consumes bandwidth exceeding a threshold amount; determining, in response to the detecting, that additional bandwidth needs to be allocated for the data message flows associated with the set of applications based on a policy that specifies allocating additional bandwidth for the data message flows when the data message flows associated with the set of applications consume more than the threshold amount; deploying a second edge forwarding element for the application set to process at least a portion of the data message flow associated with the application set so as to allocate more bandwidth to the application set, wherein before deploying the second edge forwarding element, the first edge forwarding element processes all data message flows of all application instances of the application; After the second edge forwarding element is deployed, the first edge forwarding element processes the data message flow of the first application instance set of the application, and the second edge forwarding element processes the data message flow of the second application instance set of the application.

23. A method for deploying an edge forwarding element in a software defined data center (SDDC), the method comprising: For a set of one or more applications, deploying a first edge forwarding element to process a data message flow associated with the set of applications; detecting that the data message flow associated with the set of applications consumes bandwidth exceeding a threshold amount; determining, in response to the detecting, that additional bandwidth needs to be allocated for the data message flows associated with the set of applications based on a policy that specifies allocating additional bandwidth for the data message flows when the data message flows associated with the set of applications consume more than the threshold amount; and deploying a second edge forwarding element for the application set to process at least a portion of the data message flow associated with the application set so as to allocate more bandwidth to the application set, wherein the allocated bandwidth is for data message flows between a set of machines specified by the policy in a first network of the SDDC and a subset of applications executed on machines external to the first network of the SDDC and the data message flows are data message flows entering or leaving the first network, Wherein deploying the second edge forwarding element includes (i) configuring the second edge forwarding element to forward a first subset of the portion of the data message flow to a forwarding element in an external network and (ii) configuring a set of forwarding elements in the first network to forward the first subset of the data message flow from a set of machines in the first network to the second edge forwarding element.

24. The method of claim 23, wherein the set of machines includes at least a virtual machine, a container, or a Pod.

25. The method of claim 23, wherein configuring the set of forwarding elements in the first network comprises configuring the set of forwarding elements to forward a data message flow having (i) a destination IP address associated with the second edge forwarding element and (ii) a source IP address associated with the set of machines to a second edge forwarding element.

26. The method of claim 25, further comprising configuring a gateway of the SDDC to forward the data message flow having a destination IP address associated with the set of machines to the second edge forwarding element.

27. The method of claim 23, wherein The edge forwarding elements are edge routers, and Configuring the second edge forwarding element includes configuring the second edge forwarding element to advertise a route to the set of machines to a forwarding element in the external network.

28. The method of claim 23, wherein The set of forwarding elements includes the set of intermediate routers, and Configuring the set of forwarding elements includes providing next-hop forwarding rules to the set of intervening routers.

29. The method of claim 23, wherein The set of forwarding elements includes a set of intermediate switches that implement the logical switch, and Configuring the set of forwarding elements includes providing forwarding rules to the set of intervening switches to direct the switches to forward the first set of data messages to the second edge forwarding element through the set of tunnels connecting the set of intervening switches to the second edge forwarding element.

30. The method of claim 23, wherein SDDC is a public cloud data center with a secondary network. The first network is a private network defined in the second network to implement a virtual private cloud VPC for the application set in the public cloud data center, and Deploying the second edge forwarding element includes: Deploy a gateway in a public cloud data center to serve as a second edge forwarding element for the VPC; as well as The set of forwarding elements in the second network is configured to forward a second subset of the portion of the data message flow from outside the VPC to the deployed gateway.

31. A method for deploying an edge gateway in a software defined data center (SDDC), the method comprising: receiving identities of a collection of machines operating within a first network of the SDDC and utilizing a first edge gateway to exchange data messages with a second network external to the first network; receiving a set of one or more conditions that, when satisfied, should result in dynamic deployment of a second edge gateway for the set of machines; generating a policy that identifies the set of machines and a set of conditions based on the identity; as well as The policy is distributed to a set of controllers, which evaluate the policy to determine whether the second edge gateway must be dynamically deployed to exchange data messages between the set of machines and the external second network, wherein the set of controllers collects statistical information from the set of elements in the first network, and aggregates and analyzes the collected statistical information to determine whether the set of conditions for deploying the second edge gateway has been met.

32. The method of claim 31, wherein: The receiving, generating and distributing is performed by a collection of network managers.

33. The method of claim 31, wherein the collected statistical information includes statistical information collected from the first edge gateway for sending data messages to and receiving data messages from a collection of machines.

34. The method of claim 33, wherein the first edge gateway is further configured to send and receive data messages to and from other sets of machines in the SDDC.

35. The method of claim 34, wherein the set of conditions includes at least one threshold related to a metric associated with a data message flow processed by the first edge gateway for the set of machines and other sets of machines.

36. The method of claim 35, wherein when a threshold is reached, the controller set deploys the second edge gateway and configures a set of other forwarding elements in the SDDC to forward data messages entering or leaving the SDDC and associated with the set of machines to the second edge gateway.

37. The method of claim 31, wherein the SDDC is a multi-tenant SDDC and the set of machines is a set of machines associated with tenants of the multi-tenant SDDC.

38. The method of claim 31, wherein the set of machines includes a plurality of machines executing a plurality of application instances of an application for which the second edge gateway is to be dynamically deployed.

39. The method of claim 31, wherein the collection of machines includes a plurality of machines executing a plurality of application instances of components of a multi-component application for which the second edge gateway is to be dynamically deployed.

40. A non-transitory machine-readable medium storing a program for deploying an edge gateway in a software-defined data center (SDDC), the program comprising a set of instructions for the following operations: receiving identities of a collection of machines operating within a first network of the SDDC and utilizing a first edge gateway to exchange data messages with a second network external to the first network; receiving a set of one or more conditions that, when satisfied, should result in dynamic deployment of a second edge gateway for the set of machines; generating a policy, the policy identifying the set of machines based on the set of identities and the set of conditions; as well as The policy is distributed to a set of controllers, which evaluate the policy to determine whether the second edge gateway must be dynamically deployed to exchange data messages between the set of machines and the external second network, wherein the set of controllers collects statistical information from a set of elements in the first network, and aggregates and analyzes the collected statistical information to determine whether the set of conditions for deploying the second edge gateway has been met.

41. The non-transitory machine-readable medium of claim 40, wherein the program is a program executed by a network manager.

42. The non-transitory machine-readable medium of claim 40, wherein the collected statistical information includes statistical information collected from at least the first edge gateway used to send data messages to and receive data messages from the set of machines.

43. The non-transitory machine-readable medium of claim 42, wherein the first edge gateway is further configured to send and receive data messages to and from other sets of machines in the SDDC.

44. The non-transitory machine-readable medium of claim 43, wherein the set of conditions comprises at least one threshold value related to a metric associated with a data message flow processed by the first edge gateway for the set of machines and other sets of machines.

45. The non-transitory machine-readable medium of claim 44, wherein when a threshold is reached, a controller set deploys the second edge gateway and configures a set of other forwarding elements in the SDDC to forward data messages that are entering or leaving the SDDC and are associated with the set of machines to the second edge gateway.

46. ​​The non-transitory machine-readable medium of claim 40, wherein the SDDC is a multi-tenant SDDC and the set of machines is a set of machines associated with a tenant of the multi-tenant SDDC.

47. The non-transitory machine-readable medium of claim 40, wherein the set of machines includes a plurality of machines executing a plurality of application instances of an application for which the second edge gateway is to be dynamically deployed.

48. The non-transitory machine-readable medium of claim 40, wherein the collection of machines includes a plurality of machines executing a plurality of application instances of components of a multi-component application for which the second edge gateway is to be dynamically deployed.