Network virtualization infrastructure with user role partitioning

Abstract

Some embodiments provide a method for managing a network management and control system for one or more logical networks. The method receives, from a first user, a definition of one or more security zones for a logical network. Each security zone definition includes a set of security rules for data compute nodes (DCNs) assigned to the security zone. The method receives, from a second user, a definition of an application to be deployed in the logical network. The application definition specifies a set of requirements. Based on the set of specified requirements, the method assigns DCNs that implement the application to one or more of the security zones for the logical network.

Description

Background Technology

[0001] As more and more networks move to the cloud (public, private, or hybrid), application developers within companies or other entities often want to deploy applications that can span these sites. This presents various challenges for network administrators (e.g., a company's IT team). In current systems, application developers either must understand security and networking configurations to deploy applications themselves or work with network administrators to deploy applications across the network. Various solutions are needed to address these issues. Summary of the Invention

[0002] Some embodiments of the present invention provide a network management system for managing a virtualized infrastructure having one or more logical networks defined across one or more sites (e.g., data centers). The network management system allows top-level users of the virtualized infrastructure (e.g., cloud providers, enterprise network administrators) to define the provider logical networks of the virtualized infrastructure and to define one or more secondary users (e.g., tenants, enterprise organizational units). These secondary users can then define accounts for different users. Specifically, secondary users can define security and networking configurations (e.g., logical networking constructs, security zones) that are opaque to tertiary users (e.g., application developers) and other secondary users. Application developers can specify their application requirements without any knowledge of the security and networking configurations, and the network management system in some embodiments automatically deploys applications based on these requirements while ensuring compliance with security and networking policies.

[0003] As mentioned, the top-level user defines the provider logical network (this logical network will be referred to herein as the provider logical network, regardless of whether the top-level user is a cloud provider, enterprise network administrator, etc.). In some embodiments, this provider logical network is defined using a specific data model (such as a hierarchical policy tree with nodes and connections between nodes). Through this data model, the top-level user can define security policies and security groups, as well as logical network elements (e.g., logical routers and / or logical switches). Furthermore, in some embodiments, the provider data model includes a definition of the physical infrastructure on which virtualization infrastructure is deployed. In some embodiments, this physical infrastructure may specify multiple federated sites across which the virtualization infrastructure is deployed, and groups of physical elements within these sites (e.g., groups of edge devices physically connected to external networks within each site, zones of physical servers within each site, and / or forwarding elements).

[0004] In some embodiments, secondary users (referred to herein as tenants) are granted the ability to define their own security and networking configurations and to use certain entities exposed by the provider network. While these provider network entities may be exposed to tenant users, tenant users are restricted from viewing or editing the configuration of the provider network (including these entities). Furthermore, to provide isolation between users, a tenant user is restricted from viewing (or modifying) the configurations of any other tenant user. That is, each tenant is provided with the ability to view and modify only their own configuration, and not the configurations of any other tenant or higher-level provider users. Within a tenant's configuration, sub-users may be created, who can define their own isolated configurations (in some embodiments, these configurations may be linked to the tenant or provider configuration). Each of these isolated configurations may be referred to as a Virtual Hybrid Cloud (VHC); as further described below, each VHC may be isolated to a single site or distributed across multiple physical sites (e.g., a combination of on-premises data centers, branch offices, and public clouds).

[0005] In some embodiments, the data model used for tenant configuration is similar to the data model used for provider configuration. That is, tenant configuration is also represented (e.g., in the storage device of a network management system) as a hierarchical policy tree with similar nodes and connections. However, one difference is that in some embodiments, only the provider user policy tree includes physical infrastructure nodes that define the site and groups of physical computing devices within the site. That is, some embodiments prevent tenant users from viewing or configuring provider physical infrastructure nodes and defining their own physical infrastructure policies.

[0006] Furthermore, the API is similar for tenant users and provider users when defining (e.g., creating, modifying) entities of the same type. For example, tenant administrators can define security policies and security groups, as well as logical network elements. In some embodiments, tenant users (and / or sub-users of a tenant) can define a logical network, including logical switches to which data compute nodes (e.g., virtual machines (VMs), containers, etc.) are connected, and logical routers that connect these logical switches to each other. As further described below, in some embodiments, data compute nodes (DCNs) can implement applications deployed within the tenant's logical network.

[0007] In some embodiments, a logical router may be a Layer 0 (T0) logical router or a Layer 1 (T1) logical router. The Layer 0 (T0) logical router provides connectivity to external networks, while the Layer 1 (T1) logical router (i) connects logical switches to each other, (ii) provides services (e.g., firewalls, load balancing, etc.), and (iii) enables connectivity to the T0 logical router. Some embodiments allow only provider users to define T0 logical routers so that provider users (e.g., enterprise IT teams) can oversee all rules related to external connectivity to the virtualized infrastructure (e.g., routing protocols for connecting to external routers, Virtual Private Network (VPN) connections, etc.). In some embodiments, provider users may define multiple Virtual Routers and Forwarding Tables (VRFs) for a single T0 router. On the other hand, in some embodiments, tenant users may only define T1 logical routers. In other embodiments, when a tenant user is created by the provider, an isolated VHC with a tenant-specific T0 VRF is automatically created. The tenant user can manage this T0 VRF and define services such as routing protocols (e.g., BGP, OSPF) and VPNs, while isolating this configuration from any sub-users.

[0008] In some embodiments, all types of users (providers, tenants, sub-users) can define security policies for their respective VHCs. In some embodiments, such security policies may include security rules (e.g., distributed firewall rules and / or gateway firewall rules), security group definitions, and domains. In some embodiments, a domain is a logical grouping of one or more sites (e.g., within similar geographic areas) that serves as an envelope to group different logical entities together (e.g., for security purposes). For example, in some embodiments, firewall policies or other policy microsegments applied to a domain will be automatically applied to all groups of logical endpoints defined within that domain. In some embodiments, a security group is a group of logical network endpoints that share one or more attributes (e.g., operating system, region, application layer, etc.). Users (providers or tenants) can apply security rules to security groups, and network management and control systems translate these security rules into rules applicable to each DCN within that security group. In some embodiments, security rules and groups are defined within a domain.

[0009] While tenant users cannot configure or view the configurations of entities defined within the provider's VHC, some embodiments allow provider users to expose some of these entities for tenant user use. In some embodiments, provider users expose these entities to one or more tenants via labels, which allows tenant users to identify the type of an entity and any specific characteristics imported, but not to view or modify the entity's full configuration. For example, a provider user can expose a T0 logical router, or a VRF of a T0 logical router to which only a tenant user (or their sub-user) can connect their T1 logical router. The label may indicate the sites traversed by the T0 logical router, but not the VLAN segment connected to the T0 logical router or its routing protocol configuration. Other types of entities that provider users can expose via labels include security groups that tenant users can use to define firewall rules (tenant users know the attributes that make a DCN belong to a security group, but are unaware of the full membership of groups outside the tenant's VHC), edge clusters that tenant users can use for T1 logical routers (physical infrastructure groups of devices used to implement the centralized routing components of logical routers), and services such as load balancers.

[0010] As mentioned above, in addition to being provided with the ability to configure their own isolated logical networks and security policies within the VHC, each tenant user can also define additional sub-users, who in turn can define their own isolated logical networks and security policies within the VHC. For example, if a tenant is a business line within an enterprise, then sub-users can be defined for different organizational units within that business line, and so on. Similarly, if a tenant is a public cloud tenant, then sub-users can be different business lines within the tenant scope. Furthermore, tenant users (or sub-users) can define accounts for application developers. In some embodiments, application developer users cannot view or configure their own logical networking and security rules (i.e., they are not granted their own VHC), but can define applications within the scope of a specific VHC. The implementation of these applications within the VHC will be further described below.

[0011] In some embodiments, tenant users and their sub-users can also define security zones. In some embodiments, a security zone is defined as governing the connectivity (or other security policies) of an application. That is, the DCN implementing the application is assigned to a security zone, and the connectivity and / or other security policies of that security zone are applied to those DCNs. In some embodiments, the user defining the security zone defines the span of the security zone based on the sites (of the provider's physical infrastructure), thereby restricting the applications (or application layers) assigned to that security zone to those sites. Common examples of security zones include a first zone (often referred to as a DMZ) that allows connections from external endpoints, and a second zone (often referred to as a production zone, and which may be associated with backend servers or application layers) that does not allow connections from external endpoints but allows incoming connections from the first zone.

[0012] As mentioned, application developers can be given the ability to define applications within a VHC without providing the ability to view or edit the networking and security configurations used for that VHC. In some embodiments, application developers specify a set of connectivity requirements for their applications. Based on these requirements, the network management and control system automatically assigns the DCN implementing the application to the security zone defined for that VHC.

[0013] In some embodiments, an application is defined as a set of application tiers. For example, an application is typically specified as a tier 3 application, having a web tier (a web server that accepts incoming connections from external endpoints), an application tier, and a backend database tier. Network management and control systems assign each of these defined application tiers to one of the security zones defined for the VHC based on requirements (e.g., connectivity requirements). For example, a web tier requiring incoming connections from external client endpoints would be assigned to the DMZ, while a backend database tier requiring connections from the web tier but not from external client endpoints would be assigned to the production tier. Applications can also specify connectivity requirements to other non-application DCNs (such as backup servers) within the tenant network. Connections from application DCNs are permitted as long as these non-application DCNs are assigned to the appropriate security zone (e.g., the production tier), even if these non-application DCNs reside in a different VHC than the application.

[0014] Each VHC spans a set of sites across the physical infrastructure, and in some embodiments, part of the application specification is a set of sites to which the VHC is applied (the application is deployed across this set of sites). As described above, each application tier is assigned a security zone, and this security zone spans a set of sites. Therefore, each application tier is assigned to a subset of sites specified for the application, which is the intersection of: (i) the sites specified for the application, and (ii) the sites to which the security zone for that application tier is restricted. That is, the DCN implementing the application tier is deployed within this subset of sites. If there is no overlap between the sites specified for the application and the sites to which the security zone for the application tier is restricted, some embodiments generate an error and warn the application developer user of the error so that the application developer can specify a different set of sites for the application.

[0015] As part of assigning application layers to security zones, some embodiments of the network management and control system also automatically populate various sets of firewall rules to implement a security zone for that application. For example, for internal connections (e.g., connections from the DMZ to the production area), some embodiments automatically populate distributed firewall rules to allow these specific connections. These distributed firewall rules are implemented by internal physical infrastructure (e.g., managed forwarding elements and / or firewall modules in the virtualization software of host computers), as configured by the network management and control system.

[0016] Furthermore, if the application has connectivity to an external network (e.g., if the application requests a connection from an external endpoint client), some embodiments automatically populate gateway (edge) firewall rules associated with the centralized routing component of the logical router. In some embodiments, these gateway firewall rules are arranged in sections. Firewall rules have relative priorities within sections, and each section also has different priorities, so the highest priority rule in the highest priority section is the rule with the highest overall priority. In some embodiments, the primary (highest priority) section is created and managed by the user governing the logical router (e.g., the provider for a T0 logical router, or a tenant user for a T1 logical router).

[0017] For applications requiring inbound connections, firewall rules should be configured to allow connections for that application, but not all inbound traffic at the gateway. Therefore, some embodiments have default rules that deny inbound connections, but include higher-priority rules that specify that if an inbound connection is directed to the application (e.g., based on the destination address and / or transport layer port in the packet, or determining that the packet is directed to a specific logical port on a logical router), then jump to a different application-specific firewall rule section. This application-specific firewall rule section has rules specifically configured for that application, typically along with another default rule, to deny connections that do not meet the specific requirements for that application. For example, these application firewall rules might allow any inbound connection correctly addressed to the application, thus allowing only inbound connections from certain addresses, etc.

[0018] In addition to assigning application layers to security zones and defining firewall rules for applications, some embodiments automatically generate logical networking constructs based on application definitions. Specifically, some embodiments allow users defining a VHC to define application association policies for that VHC. In some embodiments, application association policies specify how applications with layers are transformed into logical network entities. For example, application association policies may automatically define separate logical switches for each application layer, as well as T1 logical routers connecting these new logical switches. If the T1 logical router has a centralized service that requires the use of an edge gateway, then some embodiments automatically use edge devices located in an edge cluster (part of the physical infrastructure nodes of the provider policy tree) associated with the VHC in which the application is defined. If the application requires external connectivity (e.g., from an external client endpoint, or to an internal backup server located in a different VHC), then some embodiments automatically connect the T1 logical router to a specific T0 logical router exposed to that VHC. Instead of defining separate logical switches for each layer, application association policies may assign all layers (i.e., all DCNs implementing these layers) to the same logical switch. In this context, some embodiments use computational expressions to differentiate these layers (e.g., assigning tags or other metadata to DCNs that implement different layers).

[0019] The foregoing overview is intended as a brief introduction to some embodiments of the invention. It is not intended to be an introduction or overview of all the inventive subjects disclosed in this document. The following detailed description and the accompanying drawings, with reference to the detailed description, will further describe the embodiments described in the summary, as well as other embodiments. Therefore, a full review of the summary, detailed description, and drawings is necessary to understand all the embodiments described in this document. Moreover, the claimed subject matter is not limited to the illustrative details in the summary, detailed description, and drawings, but is defined by the appended claims, as the claimed subject matter can be implemented in other specific forms without departing from the spirit of the subject matter. Attached Figure Description

[0020] The novel features of the present invention are set forth in the appended claims. However, for illustrative purposes, several embodiments of the invention are illustrated in the following drawings.

[0021] Figure 1 A conceptual illustration shows an example of a hierarchical policy tree for a provider's logical network spanning multiple sites.

[0022] Figure 2 The concept illustrates the relationship between different users and their corresponding configurations.

[0023] Figure 3 A simple example of a hierarchical policy tree for a sub-user logical network is conceptually illustrated.

[0024] Figure 4 A conceptual illustration shows an example set of nested users in a network management system and their different VHCs.

[0025] Figures 5-9 The concept illustrates different methods for connecting tenant logical network structures in different VHCs to an external network via a T0 logical router and its VRF.

[0026] Figure 10 A conceptual diagram illustrates a provider policy tree, in which certain entities are exposed to tenants via labels.

[0027] Figure 11 A conceptual diagram illustrating two different tenant VHC pairs. Figure 10 The consumption of the labels shown.

[0028] Figure 12 The diagrams conceptually illustrate some embodiments of a process for deploying an application based on an application definition provided by the application developer.

[0029] Figure 13 The conceptual illustration shows the definition of two security zones for a VHC according to some embodiments.

[0030] Figure 14 The illustrations conceptually depict example application definitions according to some embodiments.

[0031] Figure 15 A conceptual illustration of what is used for Figure 14 Security zone assignment at the application layer as defined in [the document / reference].

[0032] Figure 16 The conceptual illustration depicts a process for assigning the DCN implementing the application to different sites within the security zone and virtualization infrastructure.

[0033] Figure 17 A conceptual diagram illustrates the VHC defined across four sites.

[0034] Figure 18 and Figure 19 A conceptual illustration of what is used for Figure 17 The specification defined for the application of VHC.

[0035] Figure 20 The diagrams conceptually illustrate processes for automatically generating application-related security constructs (including firewall rules) in some embodiments.

[0036] Figure 21 and Figure 22 The concept illustrates how firewall rules can be used to allow and deny different types of traffic at the gateway.

[0037] Figure 23 The diagrams conceptually illustrate some embodiments of a process for generating logical networking constructs (e.g., logical switches and / or logical routers) and their configurations based on application definitions.

[0038] Figure 24 A conceptual illustration is provided of a set of application association strategies for VHC according to some embodiments.

[0039] Figure 25 A conceptual illustration shows an example set of logical network elements defined in a policy tree used for application definitions.

[0040] Figure 26 A conceptual diagram illustrates the overall networking and security implementation of a collection of applications created within the VHC.

[0041] Figure 27 The diagrams conceptually illustrate at least a portion of the data model of a network management and control system according to some embodiments.

[0042] Figure 28 The diagrams conceptually illustrate electronic systems that utilize these systems to implement some embodiments of the present invention. Detailed Implementation

[0043] In the following detailed description of the invention, numerous details, examples, and embodiments of the invention are set forth and described. However, it will be clear and apparent to those skilled in the art that the invention is not limited to the described embodiments and can be practiced without discussing some specific details and examples.

[0044] Some embodiments of the present invention provide a network management system for managing a virtualized infrastructure having one or more logical networks defined across one or more sites (e.g., data centers). The network management system allows top-level users of the virtualized infrastructure (e.g., cloud providers, enterprise network administrators) to define the provider logical networks of the virtualized infrastructure and to define one or more secondary users (e.g., tenants, enterprise organizational units). These secondary users can further define accounts for different users. Specifically, secondary users can define security and networking configurations (e.g., logical networking constructs, security zones) that are opaque to tertiary users (e.g., application developers) and other secondary users. Application developers can specify their application requirements without any knowledge of the security and networking configurations, and the network management system in some embodiments automatically deploys applications based on these requirements while ensuring compliance with security and networking policies.

[0045] In some embodiments, these logical networks are conceptual network structures defined by an unrelated user account through a network management system. As mentioned, the top-level user defines the provider logical network (this logical network will be referred to herein as the provider logical network, regardless of whether the top-level user is a cloud provider, enterprise network administrator, etc.). In some embodiments, this provider logical network is defined using a specific data model (such as a hierarchical policy tree with nodes and connections between nodes). Through this data model, the top-level user can define security policies and security groups, as well as logical network elements (e.g., logical routers and / or logical switches). Furthermore, in some embodiments, the provider data model includes a definition of the physical infrastructure on which virtualization infrastructure is deployed. In some embodiments, this physical infrastructure may specify multiple federated sites across which virtualization infrastructure is deployed, and groups of physical elements within those sites (e.g., groups of edge devices physically connected to external networks in each site, zones of physical servers within each site, and / or forwarding elements).

[0046] Figure 1An example of such a hierarchical policy tree 100 for a provider logical network spanning multiple sites is conceptually illustrated. In some embodiments, the root node 102 represents the entire provider logical network configuration. Sections of the policy tree 100 represent logical network elements and their respective configurations, including logical forwarding elements (e.g., logical routers, logical switches, etc.). For logical network entities (e.g., logical network elements and / or policies), when a provider user account creates a new logical network entity, the network management system creates one or more nodes for that entity in the policy tree 100. As described above, these logical network entities may include logical network elements spanning one or more sites and logical network policies applicable to those elements, and connections in the policy tree represent relationships between nodes (e.g., parent-child relationships, logical network connections, etc.). Hierarchical policy trees of some embodiments, and certain types of entities represented by these policy trees, are also described in more detail in U.S. Patent Applications 16 / 906,925 and 16 / 906,944, both filed June 19, 2020, and both are incorporated herein by reference.

[0047] Logical network elements represented in the policy tree include logical forwarding elements (e.g., logical routers, logical switches, etc.). For example, in Figure 1 In this context, policy tree 100 includes Layer 0 (T0) logical routers 105, Layer 1 (T1) logical routers 110, and logical switches 145 (also referred to as overlay segments). The difference between T0 and T1 logical routers will be explained further below; in short, T0 logical routers in a provider logical network provide connectivity to external networks for the provider logical network (and in some cases, for multiple tenant logical networks), while T1 logical routers isolate a set of logical switches from the rest of the logical network and can perform stateful services for logical network endpoints connected to these logical switches.

[0048] The node used for the T0 logical router 105 has multiple child nodes, including static route definitions 130 and locale services 135 and 140 referencing different physical sites A and B. This means that the T0 logical router 105 spans these two physical sites and defines centralized routing components within these sites, as further described below with reference to the policy tree for these sites. Grandchild nodes are defined for interfaces 120 and 125 at each of these sites (e.g., uplink interfaces for external connectivity, service interfaces for third-party services) and for the IPsec VPN 115 at site B.

[0049] The node for the T1 logical router 110 has child nodes for the logical switch 145. Furthermore, this T1 logical router node 110 references the T0 logical router node 105 (dashed arrow), thereby instructing the T1 logical router 110 to connect to the T0 logical router 105 to allow logical network endpoints connected to the logical switch 145 to communicate with external endpoints (i.e., endpoints of other logical networks also connected to the T0 logical router or endpoints entirely outside the virtualization infrastructure). Although not shown for simplicity, many entities in the policy tree 100 have various additional features defining the configuration of the represented logical network entities, which can be user-defined or inherited from parent entities. For example, services such as DHCP can be defined for different logical forwarding elements, IP address ranges can be assigned to logical switches, and so on.

[0050] In addition to logical forwarding elements, the provider hierarchical policy tree 100 also includes nodes for physical entities, including nodes for each physical site (e.g., each data center in a virtualized infrastructure). For example, in Figure 1 In this diagram, nodes exist under the global root node 102 for sites A 160 and B 177. Each site has corresponding enforced point child nodes 161 and 178, under which specific physical resources, such as edge clusters, transport zones, etc., are assigned. This diagram illustrates edge clusters 151 and 152 at the grandchild nodes of site nodes 160 and 177, respectively. In some embodiments, an edge cluster is a group of edge devices that host the centralized routing components of a logical router (e.g., for implementing connectivity to external networks, providing centralized stateful services, etc.). This diagram also illustrates transport zone node 154 in site A. In some embodiments, a transport zone is a group of transport nodes in a specific part of a site (e.g., a specific rack of a server). In some embodiments, a transport node represents an entity of a physical device managed to implement a logical network. For example, in many cases, a transport node is a management program that performs virtual switches and / or virtual routers, as well as various distributed services (e.g., distributed firewalls). These managed forwarding elements and other managed network elements implement logical forwarding elements (e.g., logical switches, distributed logical routers) and various logical network policies.

[0051] In the illustrated example, edge clusters 151 and 152 have corresponding incoming references from venue services 135 and 140 attached to T0 logical router 105. In some embodiments, the edge clusters also have child nodes corresponding to a specific edge node (e.g., edge node 153). In some embodiments, when a site is added to the entire provider network represented by global root node 102, the network management system identifies the physical structure of the site (and thus the structure of the child nodes for that site in policy tree 100) via auto-discovery.

[0052] The provider network policy tree 100 also includes nodes for security-related entities. In some embodiments, security policies are organized at least in part by defining domains, which are logical groups of one or more sites (e.g., geographic regions) used as envelopes to group different logical entities together (e.g., for security purposes). Domains are defined and represented as nodes below the global root 102 in the policy tree 100. In this case, a single domain node 170 is shown in the figure. Unlike sites representing physical constructs (described below), domains are logical constructs used as envelopes to group different logical entities together (e.g., for security purposes). For example, in some embodiments, firewall policies or other policy microsegments applied to a domain will be automatically applied to all groups of logical endpoints defined within that domain.

[0053] A domain can represent a single physical site or multiple sites. In this example, domain 170 corresponds to both sites A and B, so domain enforcer node 172 references both site enforcer nodes 161 and 178. Some embodiments impose restrictions on how sites are organized into logical domains. For example, each physical site can be restricted to (i) containing only a single domain of that site and (ii) containing members of a single domain of more than one site. Furthermore, in some embodiments, a domain is created only as a top-level node below the global root node 102 (i.e., the domain is not a child domain of other domains). Provider users define domains within the provider policy tree by selecting sites that belong to that domain.

[0054] In some embodiments, logical network endpoints at each site are logically organized into security groups that can span multiple sites. These security groups are groups of logical network endpoints that share one or more attributes (e.g., operating system, region, etc.). In some embodiments, the service machine and the managed network elements running on the host computer (transmission node) apply logical network policies (such as network policy 173) to data messages exchanged between endpoints belonging to these security groups based on policy rules defined according to the security groups. In some embodiments, security groups and network policies are represented as child nodes of domains in the global policy tree 100, and accordingly inherit the span of their parent domains. In some embodiments, the span of a network policy is defined not only by its parent domain but also by the site and / or domain to which the policy references. In some embodiments, logical network policies include forwarding policies, service policies, and security policies, and in some embodiments are applied to govern the behavior of logical forwarding elements. In some embodiments, a policy may be a child node of a logical network element node (e.g., a static routing policy configuration for a logical router).

[0055] As pointed out, Figure 1 A hierarchical policy tree for a provider's logical network (i.e., the logical network of top-level users for network management and control systems) is conceptually illustrated. These top-level users are also granted the ability to create sub-users (referred to herein as tenants) and, in some embodiments, the ability to define their own security and networking configurations, as well as the ability to use certain entities exposed by the provider network. While these provider network entities can be exposed to tenant users, tenant users are restricted from viewing or editing the configuration of the provider network (including these entities). Furthermore, to provide isolation between users, a tenant user is restricted from viewing (or modifying) the configurations of any other tenant user. That is, each tenant is provided with the ability to view and modify only their own configuration, and not the configurations of any other tenant or higher-level provider users. Within a tenant's configuration, sub-users can be created, who can define their own isolated configurations (in some embodiments, these configurations may be linked to tenant or provider configurations). Each of these isolated configurations may be referred to as a Virtual Hybrid Cloud (VHC); each VHC may be isolated to a single site or distributed across multiple physical sites (e.g., a combination of on-premises data centers, branch offices, and public clouds), as further described below.

[0056] Figure 2A conceptual diagram illustrates the relationship between different users and their corresponding configurations. As shown, at the top level is configuration 205 for the provider user. In some embodiments, the network management and control system has only a single top-level provider user. This user can define logical networking and security policies as part of configuration 205. Furthermore, unlike other user configurations, provider user configuration 205 includes the physical infrastructure as part of the policy tree. Figure 1 As shown, this physical infrastructure definition can specify different sites and groups of physical computing devices at each of these sites.

[0057] Provider users can also create one or more secondary (tenant) users, providing them with initial information and constraints, and exposing certain entities from the provider's logical network to the tenant users. For example, if a provider user has multiple T0 logical routers, the provider user can associate one (or more) of these T0 logical routers (or T0 Virtual Routers and Forwarding Tables (VRFs), as described below) with a tenant. The tenant can use these T0 logical routers or T0 VRFs to provide external connectivity to (one or more) tenant logical networks. The provider also associates one or more edge clusters (of the physical infrastructure) with the tenant; in different implementations, this can be manual or automatic. The tenant can use these edge clusters to host centralized routing components of the T1 logical routers, centralized services (e.g., stateful services), etc. Furthermore, the provider user can impose certain constraints on tenant users, such as limiting the number of certain types of elements (e.g., the maximum number of T1 logical routers) or restricting the tenant logical network to a subset of sites across the entire physical infrastructure.

[0058] Tenant users can create their own configurations 210 through the network management and control system, defining logical networks and security policies. Unlike provider user configurations 205, tenant user configurations 210 (or 215) do not include any physical infrastructure; otherwise, these user configurations are structurally similar to provider user configurations. As shown in the figure, tenant users cannot view or modify provider user configurations 205 via the network management system. However, as described in more detail below, certain logical and / or physical entities of provider user configurations 205 (e.g., one or more T0 logical routers, one or more edge clusters) are exposed by the provider to tenant user configurations 210. Furthermore, tenant user configurations 210 and 215 are completely isolated from each other; that is, even if they share access to physical infrastructure resources and / or provider logical network entities, a tenant user cannot view, modify, or even need to know about any other tenant user configuration.

[0059] Furthermore, as shown in the figure, tenant users can create additional sub-users. For example... Figure 2As shown, a first tenant user (with configuration 210) creates two sub-user accounts, which in turn create their own configurations 220 and 225. Similar to the tenant user, these sub-users can also define logical networks and security in their configurations. In some embodiments, after creating these sub-users, the tenant user can provide associations to the tenant with: (i) the provider construct associated with the tenant by the provider user and (ii) any tenant user construct. For example, T0 logical router and edge cluster associations can be provided to the sub-users so that the sub-users can, for example, define T1 logical routers associated with a specific edge cluster and connect these T1 logical routers to the provider T0 logical routers without using the logical network entities of tenant configuration 210. Furthermore, any restrictions imposed on the tenant (e.g., limiting the number of certain types of logical network entities or the number of specific sites available to the tenant user) are passed on to the sub-user accounts and can be subject to further restrictions by the tenant user (e.g., if the tenant user is restricted to having a maximum of five T1 logical routers, then they can create five different sub-users and allow each of these sub-users to use only a single T1 logical router).

[0060] In some embodiments, these sub-user accounts are organized hierarchically. That is, each tenant can create one or more sub-users (referred to as organizational units in some embodiments), and these sub-users can also create further sub-user accounts. Each of the sub-user accounts at any level can create its own isolated security and networking configuration, referred to as a Virtual Hybrid Cloud (VHC) in some embodiments. In other embodiments, tenants and organizational units are containers of lower-level accounts, and only these accounts can create separate VHCs. As an example organization, a provider user can be an enterprise's IT team, on which the site implementing the virtualized infrastructure is a collection of the enterprise's main office and branch offices. In this case, a tenant user can be one or more network administrators for a business line within the enterprise, defining sub-users for different organizational units within that business line, and so on. Similarly, if a provider user is a public cloud provider with multiple data centers and a tenant is a public cloud tenant (e.g., an enterprise), then a sub-user can be a different business line within that tenant's scope.

[0061] Furthermore, some implementations allow tenants or sub-users to create application developer users. As described in more detail below, these users cannot view or modify any logical network configurations (whether it's the user who created the application developer user or any higher-level user), but they can define the requirements for applications to be deployed in a specific VHC. Figure 2As shown, the application developer user is created by one of the sub-users and provides application requirements 230, which uses the sub-user account to configure the networking and security constructs (e.g., VHC) of 220. The application developer user concept allows network administrators to enable application developers to create applications within a virtualization infrastructure without (i) providing application developers with access to the actual network configuration or (ii) manually creating networking and security constructs tailored to each application deployed in the virtualization infrastructure.

[0062] In some embodiments, the data model used for tenant configuration (or for other sub-user configuration) is similar to the data model used for provider configuration. That is, tenant configuration is also represented (e.g., in the storage device of a network management system) as a hierarchical policy tree with similar nodes and connections. Figure 3 A simplified example of a hierarchical policy tree 300 for a logical network (within a VHC) for sub-users (e.g., tenant users, sub-user accounts) is conceptually illustrated. Similar to the provider policy tree 100, the root node 305 represents the entire VHC created by the user. This policy tree 300 includes logical networking segments and security segments. However, unlike the provider policy tree 100, the sub-user policy tree 300 does not include physical infrastructure nodes that define sites and groups of physical computing devices within sites (i.e., there are no edge cluster or transport zone nodes in the sub-user policy tree). Some embodiments prevent tenant users or sub-users from viewing or configuring provider physical infrastructure nodes and defining their own physical infrastructure policies.

[0063] On the logical networking side, policy tree 300 includes a T1 logical router node 310 with three connected logical switch nodes 315-325. This T1 logical router node 310 is also shown connected to a T0 logical router node 105 as part of a provider configuration. This T0 logical router 105 can be exposed to tenant users (and thus to their sub-users), allowing users who create VHCs to connect to the T0 logical router 105 without viewing or modifying its full configuration. If the T1 logical router represented by node 310 has centralized components (e.g., for providing stateful services) rather than just distributed components, then in some embodiments, node 310 will also be associated with an edge cluster in one or more sites (e.g., via site service sub-nodes, such as...). Figure 1 (The case of T0 logical router 105 shown).

[0064] Policy tree 300 also includes security-related nodes, where domain node 330 defines domains. For simplicity, links to one or more site enforcement points (in the provider policy tree) are not shown in this diagram, but some implementations do allow tenants and other sub-users to define domains across one or more sites (although the sites used for a domain are restricted by the sites to which the VHC is limited).

[0065] In some embodiments, all types of users can define security policies for their respective VHCs. In some embodiments, this security policy may include security rules (e.g., distributed firewall rules and / or gateway firewall rules), security group definitions, etc.

[0066] For simplicity, links to one or more site enforcement points (in the provider policy tree) are not shown in this diagram, but some implementations do allow tenants and other sub-users to define domains across one or more sites (however, the sites used for the domain are restricted by the sites to which the VHC is limited). Within this domain 330, the user defines two security groups 335 and 340 and a security policy 345, which includes rules 350 referencing the two security groups 335 and 340 (e.g., rules that allow or deny traffic between group A and group B).

[0067] Each VHC in a virtualized infrastructure (i.e., a collection of data centers) managed by a network management system has a similar data model. Figure 4 A conceptual illustration shows an example set of nested users in a network management system and their different VHCs. While all VHCs are shown with the exact same structure, it should be understood that this is an illustrative example, and even though the data model is the same for all VHCs, different VHCs in a virtualization infrastructure will typically have different definitions. Furthermore, the provider infrastructure policy tree 405 is shown separately from the tenant VHCs and includes additional nodes representing the physical infrastructure nodes in the policy tree 405.

[0068] The provider creates tenant user 410 as described above, and directly within the domain of this tenant user 410, creates an organizational unit 415 and an account 420. Furthermore, within organizational unit 415, this user creates account 425. In some embodiments, any type of user can define a VHC, while in other embodiments, only the "account" user can define a VHC (as in this example). In this case, account user 425 defines two distinct, isolated VHCs 430 and 435, while account 420 defines a single VHC 440 isolated from the first two VHCs 430 and 435 (but sharing the same data model with them and the provider policy tree 405). That is, a single account can define multiple VHCs (e.g., to deploy different applications).

[0069] In some embodiments, isolation between VHCs requires that entities in one VHC cannot be associated with entities in other VHCs. That is, there should be no connection between a policy tree node for one VHC and a policy tree node for another VHC (except that they can both be associated with the same exposed entity from the provider's logical network). While VHCs are isolated, some embodiments allow some form of communication between endpoint DCNs in different VHCs (e.g., enabling a DCN implementing an application in one VHC to access backup servers in different VHCs of the same tenant or sub-user). In some embodiments, when a user creates a VHC, before defining the logical network (or otherwise), the user defines the capabilities of the VHC (e.g., the services provided within the VHC) and the capabilities of the VHC (e.g., the sites the VHC is restricted to, and various requirements related to size, scale, latency, throughput, etc.). Based on similar data models for the corresponding configurations, the API is also similar for provider users and for various sub-users when defining (e.g., creating, modifying) the same type of entities.

[0070] As mentioned above, some embodiments only allow provider users to create and configure T0 logical routers, and a specific T0 logical router or T0 VRF can be associated with different tenants and their VHCs. This allows provider users (e.g., enterprise IT teams, cloud provider network administrators) to oversee all rules related to external connectivity to the virtualized infrastructure (e.g., routing protocols used for route exchange with external routers, Virtual Private Network (VPN) connections, etc.). In some embodiments, provider users can define multiple Virtual Routers and Forwarding Tables (VRFs) for a single T0 router. On the other hand, in some embodiments, tenant users can only define T1 logical routers.

[0071] The types of logical routers in some embodiments will be described quickly and in more detail. In some embodiments, a logical router may be a T0 logical router that provides connectivity to an external network, or a T1 logical router that (i) connects logical switches to each other, (ii) provides services (e.g., firewalls, load balancers, etc.), and (iii) enables connectivity to the T0 logical router. A T0 logical router is generally defined by a network management and control system as having a distributed routing component (DR) implemented in a distributed manner and one or more centralized routing components (serving routers, or SRs) implemented in a centralized manner by edge devices. A T1 logical router may be fully distributed (e.g., if no centralized stateful service is defined for the T1 logical router), or may similarly have both a DR and one or more SRs. Details of these types of logical routers and logical routing components are further described in U.S. Patent Application 16 / 906,905, filed June 19, 2020, which is incorporated herein by reference.

[0072] In some embodiments, the provider may expose external connectivity via a T0 logical router to the tenant VHC in various ways. As described above, some embodiments allow defining one or more VRFs (i.e., independent routing tables to which packets can be assigned) for a particular T0 logical router, and allow these VRFs to be exposed to tenant users (or the T0 VRFs can be managed by tenant users) rather than exposing the T0 logical router itself. In some embodiments, the SR for the T0 logical router is configured to implement all VRFs for this T0 logical router, regardless of whether these VRFs are managed by provider users or tenant users. When a particular T1 logical router connects to a particular T0 VRF, outgoing packets processed by that particular T1 logical router are sent to the T0 logical router in some embodiments (e.g., based on an internally defined logical switch for connecting two logical routers) with an indicator (within the encapsulation header) indicating that the packet originated from that particular T1 logical router. Based on this indicator, the T0 SR applies the correct VRF to these packets.

[0073] Figures 5-9 The diagrams conceptually illustrate different methods for constructing logical networking connections between tenants in different VHCs and external networks via a T0 logical router and its VRF. For example, in Figure 5In this example, two different VHCs (potentially belonging to different accounts) managed by the same tenant share the same T0 VRF. The provider defines both a T0 gateway logical router 500 and a VRF 505 for this logical router, exposing the VRF 505 to the tenant user. This tenant user manages two VHCs, 510 and 515, with corresponding T1 logical routers 520 and 525 defined. In this example, both T1 logical routers 520 and 525 (in different VHCs) are connected to the T0 VRF 505.

[0074] Figure 6 This conceptual illustration shows that, in some embodiments, two different VHCs managed by different tenants can also connect to the same VRF if that VRF is exposed to both tenants by the provider. In this example, the provider again defines a T0 gateway logical router 600 and a VRF 605 for this logical router, exposing VRF 605 to tenant users A and B. Each of these tenant users manages their respective VHCs 610 and 615, in which corresponding T1 logical routers 620 and 625 are defined. In this example, both T1 logical routers 620 and 625 (in different VHCs managed by different tenants) are connected to T0VRF 605. It should be noted that if two or more different tenants use the same T0VRF exposed by the provider, the provider should ensure that the IP addresses assigned to the different tenants do not overlap, as any overlap can cause confusion for the T0 VRF when handling incoming data traffic.

[0075] Figure 7 This conceptual illustration shows how a provider can expose different VRFs of the same T0 logical router to different tenants, allowing VHCs managed by these tenants to connect to different VRFs. In this example, the provider defines a T0 gateway logical router 700 and two VRFs 705 and 710 for this logical router. The first VRF 705 is exposed to tenant user A, while the second VRF 710 is exposed to tenant user B. Each of these tenant users manages their respective VHCs 715 and 720, in which corresponding T1 logical routers 725 and 730 are defined. These T1 logical routers 725 and 730 are connected to their respective T0 VRFs 705 and 710, which are exposed by the provider users to their respective tenants.

[0076] Figure 8This conceptual illustration shows one or more tenants managing VRFs associated with a T0 logical router within their own VHC (or within different VHCs managed by the tenant, depending on the configuration). In this example, the provider defines a T0 gateway logical router 800 and exposes this logical router 800 to tenant users A and B. Each of these tenant users manages their respective VHCs 815 and 820, in which corresponding T1 logical routers 825 and 830 are defined. Furthermore, each of the tenant users manages corresponding T0 VRFs 805 and 810. These VRFs can be automatically defined when a tenant user is created, or they can be created individually and linked to the T0 logical router 800 in different embodiments. While the preceding example illustrates a scenario where the provider fully manages the T0 logical routers and T0 VRFs, in this example, the T0 VRFs are created as tenant-specific and can be managed by that tenant. It should be noted that while this example illustrates T0 VRFs 805 and 810 and T1 logical routers 825 and 830, as well as logical switches, within a VHC, in some embodiments, the tenant-managed T0 VRF is automatically created for the tenant in an isolated VHC (i.e., separate from any other VHCs the tenant may have). Tenant users can manage this T0 VRF and define services such as routing protocols (e.g., BGP, OSPF) and VPNs. If the T0 VRF is configured in a separate VHC, this configuration is isolated from any sub-users; however, tenant users can also expose the T0 VRF to their sub-users for use with T1 logical routers connected to their VHC.

[0077] at last, Figure 9 This conceptual illustration shows that a provider can simply expose the T0 logical router itself to a tenant (or multiple tenants) without using a VRF at all (although in some such cases, the actual edge device implementing the SR for the T0 logical router may use one or more VRFs to handle routing). In this example, the provider defines a T0 gateway logical router 900 and exposes this logical router to a tenant user. This tenant user manages two VHCs, 905 and 910, in which corresponding T1 logical routers 915 and 920 are defined. In this example, both T1 logical routers 915 and 920 (in different VHCs) are directly connected to the exposed T0 logical router 900.

[0078] While tenant users cannot configure or view the configuration of entities defined within the provider's logical network, as indicated, some embodiments allow provider users to expose some of these entities for tenant user use. In some embodiments, provider users expose these entities to one or more tenants via tags, which allows tenant users to identify the type of the entity and any specific characteristics imported, but not to view or modify the entity's full configuration.

[0079] Figure 10 A conceptual illustration of a provider policy tree 1000 is provided, in which certain entities are exposed to tenants via tags. Similar to... Figure 1 The policy tree shown, provider policy tree 1000, includes a T0 logical router node 1005, which (in addition to other child and grandchild nodes) defines a T0 VRF node 1010. Policy tree 1000 also includes a node 1015 representing a domain, which includes security group 1020 and defined network policies 1025. Finally, the physical infrastructure portion of policy tree 1000 includes site 1030, which (under the site enforcement point) has grandchild nodes for transport zone 1032 and edge cluster 1035.

[0080] The diagram also illustrates how several of these entities are exposed to tenants via labels. Specifically, T0 logical router 1005 is exposed via label 1040, T0 VRF 1010 via label 1045, edge cluster 1035 via label 1050, and security group 1020 via label 1055. The use of labels allows providers to expose an entity without revealing its configuration (i.e., tenants have no way of seeing the configuration of the exposed entity). T0 logical router label 1040 or T0 VRF label 1045 enables tenant users (or their sub-users) who expose these labels to connect their T1 logical routers to these entities without needing to see the logical router configuration. Such labels may indicate the sites spanned by the T0 logical router, but not the VLAN segments connected to the T0 logical router or their routing protocol configuration. Edge cluster label 1050 may indicate the site where the edge cluster resides, but not any details available to the provider about the edge cluster (e.g., the type or number of edge devices in the cluster). Finally, security group label 1055 can indicate the attributes that make DCN belong to a security group, but does not indicate the full membership of that group outside the tenant VHC. It should be noted that in some embodiments, other types of entities (e.g., services such as load balancers) can be exposed via labels.

[0081] Figure 11The diagram conceptually illustrates the consumption of these labels 1040-1055 by two different tenant VHCs. For simplicity, only the relevant nodes of policy trees 1105 and 1110 for these VHCs are shown. The first VHC, represented by root node 1115, includes a T1 logical router represented by node 1120. This T1 logical router 1120 is purely distributed (i.e., there are no services requiring a centralized routing component for T1 logical router 1120) and therefore does not require an edge cluster. However, T1 logical router 1120 is connected to a T0 logical router 1005 in the provider network via label 1040. The second VHC, represented by root node 1125, includes a T1 logical router represented by node 1130 and a domain represented by 1135. Here, T1 logical router 1130 requires a centralized routing component and is therefore linked to edge cluster 1035 via label 1050. Furthermore, T1 logical router 1130 is connected to T0 VRF 1010 via label 1045. In the first VHC, the T1 logical router 1120 is directly connected to the T0 logical router 1005, while in the second VHC, the T1 logical router 1130 is connected to the T0 VRF 1010. This is because, in this case, the provider exposes the T0 label 1040 to the tenant managing the first VHC and the T0 VRF label 1045 to the tenant managing the second VHC. Finally, domain 1135 in the represented second VHC 1110 consumes security group 1020 via label 1055, such that security group 1055 is part of domain 1135 defined within the second VHC.

[0082] In addition to the user types described above, in some embodiments, tenant users or sub-users can define user accounts for application developers. In some embodiments, these application developer users are associated with a VHC but cannot view or configure the logical networking and security rules of that VHC (i.e., these users are not granted the ability to create new VHCs or view the configuration of the VHCs they are associated with). More precisely, these users can define applications within the scope of the associated VHC, and various policies defined for the VHC enable the automatic implementation of these applications without additional intervention from the user who owns the VHC.

[0083] Figure 12 A process 1200 for deploying an application based on an application definition from an application developer user is conceptually illustrated in some embodiments. In some embodiments, process 1200 is executed by a network management and control system upon receiving an application definition for a specific VHC. Various different operations of process 1200 will be described in more detail below. It should be understood that process 1200 is a conceptual process, and the operations shown in this process may be executed in different orders or in parallel.

[0084] As shown in the figure, process 1200 begins by receiving (at 1205) the definition of an application to be deployed in a specific VHC. In some embodiments, the application definition specifies the deployment requirements for a real-world application. These applications can include widely used applications such as SharePoint, Epic, or SAP, as well as business-specific single-tier or multi-tier proprietary applications. The requirements specified by the application definition can include, for example, connectivity requirements (e.g., whether external connectivity is required), service requirements (e.g., load balancers, etc.), and the set of available sites in which the application can be deployed (e.g., if the application does not comply with the European General Data Protection Regulation (GDPR), then it can only be deployed on North American sites), and other requirements. In some embodiments, as further described below, the application is defined based on a set of layers (e.g., web layer, application layer, database layer) and the connectivity and other requirements of each layer.

[0085] Process 1200 then determines (at 1210) the security zone to which the application is deployed. In some embodiments, tenant users and their sub-users (i.e., the owners of the VHC) can define security zones within the VHC. In some embodiments, a security zone is defined as governing the application's connectivity (or other security policies). That is, the DCN implementing the application or application layer is assigned to a security zone, and the connectivity and / or other security policies of the security zone are applied to these DCNs. Examples of common security zones include a first zone (often referred to as a DMZ) that allows connections from external endpoints and a second zone (often referred to as a production zone, and which may be associated with backend servers or application layers) that does not allow connections from external endpoints but allows incoming connections from the first zone. Some embodiments may also include a test security zone for the application during the testing phase (when minimal or no external traffic is required).

[0086] Next, process 1200 determines (at 1215) the sites where the application will be deployed. In some embodiments, the user defining the security zone also defines the span of the security zone in terms of sites (of the provider's physical infrastructure), thereby restricting the applications (or tiers of applications) assigned to the security zone to these sites. In some embodiments, the sites to which the security zone is restricted may include any sites to which a containing VHC is restricted, but may not include other sites that are not available to that VHC. The application may be deployed in (i) any site specified for the application and (ii) any site available to the security zone(s) to which the application is assigned. In some embodiments, this determination is made on a tiered basis.

[0087] Process 1200 also determines (at 1220) the logical networking setup for the application. Some embodiments automatically generate the logical networking construct based on the application definition. Specifically, some embodiments allow the user defining the VHC to define application association policies for the VHC. In some embodiments, the application association policy specifies how to convert an application into a logical network entity. For applications with multiple layers (which may also be referred to as application subnets in some cases), the application association policy may automatically define a separate logical switch for each application layer, as well as a T1 logical router connecting these new logical switches. Instead of defining a separate logical switch for each layer, the application association policy may also assign all layers (i.e., all DCNs implementing these layers) to the same logical switch. In this case, some embodiments use computational expressions to distinguish these layers (e.g., assigning tags or other metadata to DCNs implementing different layers).

[0088] Process 1200 also generates (at 1225) firewall rules for traffic to and from the application. In some embodiments, these firewall rules define the security zones to which the application is assigned for traffic to and from the application. For example, for connections within a virtualization infrastructure (e.g., for connections between a DCN assigned to the DMZ and a DCN assigned to the production area), some embodiments automatically populate distributed firewall rules to allow these specific connections.

[0089] Furthermore, if the application has connectivity to an external network, some embodiments automatically populate gateway (edge) firewall rules to ensure these connections are allowed. In some embodiments, these gateway firewall rules are arranged in segments. Firewall rules have relative priorities within segments, and these segments also have different priorities, so the highest priority rule in the highest priority segment is the rule with the highest overall priority. For applications requiring incoming connections, firewall rules are configured to allow connections for this application to come in, but not to allow all incoming traffic at the gateway. Therefore, some embodiments have a default rule to deny incoming connections, but include a higher priority rule that specifies that if an incoming connection is directed to the application, it should be redirected to a different firewall rule segment specific to that application. The application-specific firewall rule segment has rules specifically configured for that application, typically along with another default rule, to deny connections that do not meet the specific requirements for that application.

[0090] Next, process 1200 deploys (at 1230) the DCN implementing the application at the identified site. In some embodiments, this operation is actually performed by a compute manager that works separately from but in conjunction with the network management system. The DCN, in various embodiments, may be a VM, container, or other type of DCN, and is deployed with the software required to implement the application (or a specific layer of the application). In some embodiments, this deployment also enables these DCNs to be attached to the appropriate logical ports of the logical switches (or other logical forwarding elements) to which the DCNs are connected, according to the logical networking configuration determined for the application.

[0091] Finally, process 1200 configures (at 1235) the managed network elements to implement logical networking and firewall rules (and any other security or other services required by the application). As described above, these managed network elements include various components that execute in the node's management program to perform logical switching, logical routing, implementations of distributed firewall rules, etc. The managed network elements also include edge devices that implement T0 and T1 SRs and, in some embodiments, gateway firewall rules. Process 1200 then concludes.

[0092] As described above, in some embodiments, tenant users and their sub-users can define security zones through a network management system. These security zone governance applications apply connectivity or other security policies, such that connectivity and / or other security policies for the security zone are applied to the DCN assigned to that security zone.

[0093] Figure 13 A conceptual illustration shows the definitions of two security zones 1300 for a VHC according to some embodiments. These two security zone definitions include a DMZ definition 1305 and a production zone definition 1310. As shown, DMZ definition 1305 specifies that applications (or application layers) assigned to the DMZ are allowed to receive ingress connections from external sources (e.g., from web clients). This connectivity is important for applications that allow users to interact with the application via the Internet (e.g., without a VPN connection). Furthermore, DMZ definition 1305 specifies that egress connections to the DCN assigned to the production zone are allowed. Similarly, production zone definition 1310 specifies that ingress connections from the DMZ are allowed, but connections to external sources are denied. While only two security zones are specified in this example, it should be understood that other types of security zones may be specified in some embodiments; for example, a test security zone may not allow connections from outside the test zone (i.e., for use by the application during the testing phase).

[0094] It should be noted that in this example, the security zone is defined at the same level for a specific VHC. In some embodiments, the network management system allows a user (e.g., a tenant user or sub-user, or even a provider user) to define the security zones available across all VHCs under that particular user. That is, a provider user can define a DMZ security zone for all VHCs throughout the virtualization infrastructure. Similarly, a tenant can define a DMZ security zone for all VHCs belonging to that tenant and its sub-users (if the provider has not defined one).

[0095] While the security zone applicable to a VHC is defined by the VHC owner or a higher-level user, the applications are defined by the application developer user. In some embodiments, these applications are defined as a collection of application layers, with requirements specified for each layer. For example, an application is typically specified as a 3-tier application, having a web layer (a web server that accepts incoming connections from external endpoints), an application layer, and a backend database layer.

[0096] Figure 14 A conceptual illustration of an example application definition 1400 according to some embodiments is provided. Instead of a traditional three-tier application, application definition 1400 specifies two layers. These two layers are a web layer 1405 and a backend layer 1410, and each layer specifies its requirements. In some embodiments, from the application developer's perspective, the application layer is a collection of workloads (e.g., DCNs) sharing public networking and security services. Application definition 1400 specifies these required services for each layer. In some embodiments, connectivity requirements are also specified as services. In this case, web layer 1405 specifies requirements for external connectivity (i.e., allowing access from external clients) and load balancing across the DCNs. Backend layer 1410 specifies requirements for connectivity with web layer 1405 (i.e., allowing access from the web layer DCN), but does not specify any other services. Some embodiments may specify additional services such as web application firewalls (WAFs), intrusion detection and / or protection systems (IDS / IPS), etc.

[0097] In some embodiments, certain application service constructs are associated with the VHC, and application developers can choose from these services when specifying the requirements for the application to be deployed in the VHC. These application service constructs may be exposed to application developers as templates that indicate the configuration for a particular service (e.g., load balancing, etc.) and combine (i) pre-populated configuration elements that are independent of the application specification, (ii) configuration elements populated based on application characteristics, and (iii) configuration elements populated based on explicit application developer input.

[0098] Upon receiving an application definition that specifies one or more application layers for a particular VHC, the network management and control system assigns each of these defined application layers to one of the security zones defined for that particular VHC based on the application requirements (e.g., connectivity requirements). Figure 15 A conceptual illustration shows the security zone assignment 1500 for application layers 1405 and 1410. In this example, web layer 1405, which requires incoming connections from external client endpoints, is assigned to DMZ 1305. Backend layer 1410, which requires connectivity to web layer 1410 but should not have any connectivity to external client endpoints, is assigned to production zone 1310. It should be noted that application layers can also specify connectivity requirements to other non-application DCNs (such as a collection of backup servers) within the tenant's logical network. Connections from application DCNs are permitted, even if these non-application DCNs are located in different VHCs than the application, as long as these non-application DCNs are assigned to the appropriate security zone (e.g., production zone).

[0099] Each VHC is defined as a set of sites spanning the physical infrastructure, and in some embodiments, part of the application specification is a set of sites across which the application's VHC can be deployed. Furthermore, the span of a security zone is defined by the VHC owner (or other higher-level user) regarding the set of sites to which the security zone is restricted, thereby restricting the application layer assigned to that security zone to those sites. Therefore, each application layer is assigned to a subset of sites specified for that application, which is the intersection of: (i) the sites specified for the application, and (ii) the sites to which the security zone for that application layer is restricted. That is, the DCN implementing the application layer is deployed within this subset of sites to which the application layer is assigned. If there is no overlap between the sites specified for the application and the sites to which the security zone for the layer used for that application is restricted, some embodiments generate an error and warn the application developer user of the error so that the application developer can specify a different set of sites for the application.

[0100] Figure 16 The diagram conceptually illustrates a process 1600 for assigning a DCN (Distributed Control Network) implementing an application to different sites within a security zone and virtualization infrastructure, representing several embodiments. In some embodiments, process 1600 is primarily executed by a network management and control system upon receiving the application definition; however, as mentioned above, the actual deployment of the DCN can be performed by a compute manager (e.g., one that works in conjunction with the network management system). Further details will be provided by reference. Figure 17-19 Partially describe process 1600, Figure 17-19 The diagram illustrates the distribution of security zones across sites associated with the VHC and the assignment of example application layers to these sites.

[0101] Specifically, Figure 17 The conceptual diagram illustrates that VHC 1700 is defined as spanning four sites 1-4. For this purpose, VHC 1700 defines two security zones, DMZ 1705 and Production Zone 1710. DMZ 1705 is confined to sites 1 and 2, while Production Zone 1710 is confined to sites 2-4. These security zones 1705-1710 are defined according to... Figure 13 The specification definition shown will be used for Figure 18-19 The example shown is to describe process 1600 more fully.

[0102] As shown in the figure, process 1600 begins by receiving (at 1605) the definition of an application to be deployed in a specific VHC. In some embodiments, the application definition specifies the layers of the application, the requirements for each layer of the application, and the set of sites across which the application can be deployed. In some embodiments, the sites specified for the application may include only the sites associated with the specific VHC (e.g., in...). Figure 17 In this case, only sites 1-4 can be included in any application definition used for VHC 1700. As mentioned above, applications can include widely used applications as well as proprietary applications specific to business requirements. Requirements specified by the application definition can include connectivity requirements (e.g., whether external connectivity is required) and / or other service requirements (e.g., load balancers, etc.).

[0103] Figure 18 and 19 The specifications for application definitions 1800 and 1900 are conceptually illustrated. Application definition 1800 specifies the web layer 1805 and the backend layer 1810, and indicates that the application can be deployed on sites 2 and 3. Similarly, application definition 1900 specifies the web layer 1905 and the backend layer 1910, and indicates that the application can be deployed on sites 3 and 4.

[0104] Return to Figure 16 In process 1600, one of the specified application layers is selected (at 1610) to assign the DCN of this application layer to a security zone and identify the site for that application layer. It should be understood that while process 1600 evaluates these layers sequentially, one by one, this is merely a conceptual process. In some embodiments, actual network management and control systems evaluate all layers in parallel to assign them to security zones and determine the site for each layer.

[0105] Procedure 1600 assigns the selected layer (at 1615) to the security area based on the layer's requirements. In both examples, as... Figure 14As shown, based on the layer requirements (i.e., the web layer requires external connectivity, while the backend layer requires connectivity with the web layer but not external connectivity), web layers 1805 and 1905 are assigned to DMZ 1705, while backend layers 1810 and 1910 are assigned to production zone 1710.

[0106] Next, to determine the site for the selected layer, process 1600 identifies (at 1620) the intersection of the following: (i) the site specified for the application, and (ii) the site to which the security zone assigned to the selected application layer is restricted. That is, each application layer is restricted by two factors: the DCN implementing this layer can only be deployed in the site specified by (i) the application developer for the application and (ii) the site to which the layer is assigned by the user defining the security zone (e.g., the user managing the VHC).

[0107] Based on this identification, process 1600 determines (at 1625) whether the intersection includes any stations (i.e., whether the intersection is empty). If no stations satisfy either of these criteria, then process 1600 (at 1630) generates an error. Figure 19 In this case, web layer 1905 was assigned to DMZ 1705. However, the application definition specified that the application should be deployed in sites 3 and 4, while the DMZ only spans sites 1 and 2. Therefore, an error was returned to the application developer (e.g., indicating that the web layer connectivity requirements in the application specification require the application to include at least one of sites 1 and 2).

[0108] On the other hand, if at least one site meets two criteria for the selected layer, then process 1600 will assign (at 1635) the workload (e.g., DCN) for that layer to the identified site(s). Figure 18 The illustration shows that web layer 1805 is assigned only to site 2, while backend layer 1810 is stretched across sites 2 and 3. This is because web layer 1805 is assigned to DMZ 1705, which is restricted to sites 1 and 2 (the application is only assigned to site 2 among these sites). Meanwhile, backend layer 1810 is assigned to production zone 1710, which is restricted to sites 2-4 (where the application is assigned to both sites 2 and 3 among these sites). Some embodiments always stretch each layer across all sites to which it was assigned, while other embodiments employ various optimization algorithms to determine whether each layer should be stretched across multiple available sites. For example, where possible, some embodiments attempt to assign workloads to sites where all layers of the application overlap (e.g., ...). Figure 18 (Site 2 in the example). Other embodiments allow each tier to be stretched across all its assigned sites, but use traffic optimization to determine which site each actual DCN should be deployed to.

[0109] Next, process 1600 determines (at 1640) whether any additional layers still need to be evaluated. If there are additional layers, the process returns to 1610 to select the next layer. As mentioned earlier, process 1600 is a conceptual process, and some embodiments actually evaluate all layers simultaneously.

[0110] Once all layers have been evaluated (and no errors have been generated), process 1600 deploys (at 1645) the DCN implementing each layer at the site to which the layer workload is assigned. In some embodiments, this deployment is actually performed by a compute manager that works separately from but in conjunction with the network management system. The DCN, in various embodiments, can be a VM, container, or other type of DCN and is deployed with the software required to implement the corresponding application layer. In some embodiments, this deployment also enables these DCNs to be attached to the appropriate logical ports of the logical switches (or other logical forwarding elements) to which the DCNs are connected, based on the logical networking configuration determined for the application, as described in more detail below.

[0111] As part of the process of assigning application layers to security zones, some embodiments of the network management and control system automatically generate various sets of firewall rules to implement security zones for applications. These firewall rules enforce the ability or inability of DCNs in different security zones to communicate with each other and / or with external sources. In some embodiments, the firewall rules defined for implementing security zones for applications include distributed firewall rules (enforced by managed forwarding elements and / or firewall modules executed in the virtualization software of the host computer) and gateway firewall rules (enforced by edge devices or firewalls connected to edge devices).

[0112] Figure 20 The diagram conceptually illustrates a process 2000 for automatically generating application-related security constructs (including firewall rules) according to several embodiments. In some embodiments, process 2000 is primarily executed by a network management and control system upon receiving an application definition. It should also be understood that process 2000 describes only one possible set of security constructs that can be defined by the network management system to enforce security zones assigned to it, and other similar processes defining similar sets of security constructs are possible in different embodiments.

[0113] As shown in the figure, process 2000 begins by receiving (at 2005) the application definition, which specifies the application's layers and the connectivity (and other) requirements for each layer. As mentioned earlier, some of these requirements are specified for the application services that the application will consume. In some embodiments, the application developer is required to provide values ​​for certain services based on an application service template.

[0114] Procedure 2000 then assigns the layer (at 2010) to the security area, as described above (e.g., by referring to...). Figure 16 These assignments are based, at least in part, on the connectivity requirements specified for each layer. Multiple layers of an application can be assigned to the same security zone (e.g., for a 3-tier application, both the application layer and the database layer can be assigned to the production security zone). Based on these assignments and application definitions (e.g., different layers), Process 2000 generates various security constructs.

[0115] Procedure 2000 (at 2015) defines a domain for the application within the policy tree of the application's VHC. As described above, in some embodiments, a domain is a logical construct used as an envelope for various security groups and policies. While the procedure 2000 described herein defines one domain for the application (with all security groups associated with the application), some embodiments define different domains for each service specified by the application.

[0116] In this context, within the domain, Procedure 2000 defines (at 2020) security groups for: (i) the application, and (ii) each layer of the application. A given DCN can belong to many different security groups. For example, a DCN that is part of the web layer of an application can belong to a first security group for any DCN that implements the application, a second security group for the DCN that implements the web layer of the application, a third security group for any DCN that runs a particular operating system, a fourth security group for DCNs assigned to the DMZ (assuming the web layer is assigned to this security zone), and so on.

[0117] Procedure 2000 also defines (at 2025) distributed firewall rules to allow (i) communication between different application layers that need to be able to communicate with each other, and (ii) communication between an application layer and other internal DCNs (if such communication is necessary). In some embodiments, distributed firewall rules specify whether to allow or deny specific traffic (e.g., from one or more source security groups to one or more destination security groups). Specifically, some embodiments use a whitelisting scheme, where it is assumed that all traffic is blocked unless specifically allowed by a firewall rule (e.g., by having a default rule that denies traffic that does not conform to any other rule). As an example of such a rule, a distributed firewall could be used to allow traffic from an application's web layer to its database layer, specifying that traffic originating from a web layer security group and destined for a database layer security group is allowed. Furthermore, some applications may require connectivity to a DCN within the virtualization infrastructure. For example, a particular tenant might set up a backup server for use by multiple applications within a VHC or a collection of VHCs. In some embodiments, these backup servers (or other services with which multiple applications communicate) are created as separate applications, and distributed firewall rules are defined to allow traffic between the backup servers and application layers that require connectivity to the backup servers. In some embodiments, each connectivity requirement is specified for an application service, which in turn specifies one or more distributed (or gateway) firewall rules based on application details.

[0118] Furthermore, process 2000 determines (at 2030) whether connectivity to an external source is required. In some embodiments, this connectivity is specified as an application service requirement for a specific layer of the application and invokes the creation of specific firewall rules. Because this traffic passes through the T0 logical router SR (i.e., the gateway at the edge device), some embodiments generate gateway (edge) firewall rules (i.e., firewall rules associated with the SR) to allow certain traffic to (and from) the application.

[0119] If connectivity to external sources is required, the process defines (at 2035) firewall rule application segments (i.e., segments of firewall rules specific to the application) to allow specific traffic from the external source to the application. In some embodiments, gateway firewall rules (and separate distributed firewall rules) are arranged by segments. Firewall rules have relative priorities within segments, and segments also have different priorities, so the highest priority rule in the highest priority segment is the rule with the highest overall priority. In some embodiments, the primary (highest priority) segment is created and managed by the user governing the logical router (e.g., the provider for a T0 logical router, or a tenant user for a T1 logical router). Furthermore, some embodiments generate segments for each security zone (e.g., segments for the DMZ, segments for the production zone, etc.), as well as a separate (lower priority) segment for the application. Within this application segment, process 2000 adds rules that allow specific traffic from external sources (e.g., from the Internet, from any IP address, etc.) to be directed to the application. In some cases, this rule can use the public IP address used by the application instead of the application's actual web layer IP address, because this public IP address will be used in traffic processed by the T0 gateway (before performing network address translation on the traffic). Rules allowing certain traffic can also be configured to allow connections only from certain addresses, depending on the specified connectivity requirements for the application. Furthermore, application segments typically include a default rule that specifies that any additional traffic that does not meet the specified criteria will be rejected.

[0120] Procedure 2000 also defines gateway firewall rules (at 2040) in the main segment that hop to the application segment for traffic originating from an external source directed to the application. This main segment can be a highest-priority application segment managed by the provider as part of a T0 gateway, or it can be a segment associated with a specific security zone (e.g., a DMZ) receiving incoming connections. That is, this main firewall rule segment should not be configured to allow all incoming traffic at the gateway and therefore have a default rule that denies traffic. Instead of specifically allowing high-priority traffic, rules are generated with a new type of action that skips the rest of the segment (and hops to the application segment) to redirect data traffic directed to the application to a lower-priority application firewall rule segment. Therefore, the details of allowing or denying such incoming traffic are handled by the rules generated for this segment (at 2035). This new rule can identify incoming traffic for the application based on the destination address and / or transport layer port in the packet, or the determination that the packet is directed to a specific logical port of the logical router associated with the application.

[0121] After generating the required firewall rules for the application, process 2000 configures (at 2045) the managed network elements to implement the defined firewall rules. As previously described, distributed firewall rules are implemented by internal physical infrastructure (e.g., managed forwarding elements and / or firewall modules in the virtualization software of host computers) in some embodiments, and the network management and control system configures these elements to use the newly generated firewall rules. Similarly, in some embodiments, edge devices implementing TO SR also implement gateway firewall rules, and the network management and control system also configures these edge devices and / or their associated edge firewalls.

[0122] Figure 21 and Figure 22 The diagram conceptually illustrates the use of these rules to allow and deny different traffic at gateway 2100. As shown, gateway 2100 (in addition to implementing various logical forwarding elements and other services) implements a gateway firewall. In different embodiments, gateway firewall 2105 may be implemented by the same data path, different elements, or even a separate (e.g., third-party) device connected to the physical gateway edge device 2100, implementing logical forwarding elements.

[0123] Gateway firewall 2105 includes firewall rules in at least two segments. The main segment 2110 includes a first rule specifying that if the destination IP address of a data message is "App.Public.IP" (i.e., a public IP address used by a specific application), the action to be taken is to jump to application segment 2115 (without making a final decision on whether to allow or deny the packet). Main segment 2110 also includes a second rule to deny any other incoming data messages. Application segment 2115 includes a first rule specifying that if the destination IP address of a data message is "App.Public.IP" and the destination transport layer port is "K", then the data message should be allowed. That is, the application connectivity requirement translated into firewall rules requires that data messages directed to the application also specify the correct transport layer port K. Segment 2115 also includes a default deny rule.

[0124] exist Figure 21 In this process, gateway 2100 receives data message 2120 with source IP address A, destination IP address "App.Public.IP", source transport layer port J, and destination transport layer port K. During the processing of data message 2120, gateway firewall 2105 first checks the main segment 2110, and data message 2120 matches the first rule. Therefore, gateway firewall 2105 jumps to application segment 2115. Here, because data message 2120 is correctly directed to the application (using the correct IP address and port number), gateway firewall 2105 allows the data message.

[0125] exist Figure 22 In this process, gateway 2100 receives a second data message 2200 with source IP address A, destination IP address B, source transport layer port J, and destination transport layer port K. During the processing of data message 2200, gateway firewall 2105 first checks primary segment 2110. Here, data message 2200 does not match the first rule, so the second rule is applied to reject the data message. Therefore, only data messages directed to the application IP address are processed by application segment 2115; other data messages are rejected by primary segment 2110.

[0126] In addition to assigning application layers to security zones and defining firewall rules for applications, some embodiments automatically generate logical networking constructs based on application definitions. Specifically, some embodiments allow users (or higher-level users) managing the VHC to define application association policies for the VHC. In some embodiments, these application association policies specify how application definitions with one or more layers are translated into logical networking constructs (e.g., logical routers, logical switches, etc.).

[0127] Figure 23 The diagram conceptually illustrates a process 2300 for generating logical networking constructs (e.g., logical switches and / or logical routers) and their configurations based on application definitions, according to several embodiments. In some embodiments, process 2300 is executed by a network management and control system upon receiving application definitions from an application developer user. Reference will be made, at least in part, to... Figure 24 and Figure 25 To describe process 2300, Figure 24 A conceptual diagram illustrates a set of application association strategies. Figure 25 The concept illustrates the use of application association strategies to generate logical forwarding elements based on application definitions.

[0128] As shown in the figure, process 2300 begins by receiving (at 2305) the definition of an application to be deployed in the VHC, which specifies one or more layers and connectivity requirements (and other types of requirements) for each layer. As described above, in some embodiments, the application definition specifies the number of layers, connectivity and other service requirements for each layer, and other information. In some embodiments, if the application is a generic application type (e.g., Epic, SharePoint, etc.), the application developer can use a template provided by the network management system for the application (which uses a standard set of layers and requirements for the application type) with minimal customization. If the application is a proprietary application, some embodiments require additional customization from the application developer (e.g., the number of layers, specific connectivity requirements).

[0129] Based on the application definition, process 2300 determines (at 2310) the appropriate application association policy for the VHC in which the application is deployed, for use in generating a logical networking construct for the application. In some embodiments, the VHC manager associates one or more application association policies (which may be predefined by the system or generated by the user managing the VHC) with the VHC. Different application association policies may specify different logical networking constructs based on the number of layers in the application (e.g., different policies for 2-layer and 3-layer applications), whether the application is a specific type of application (e.g., there may be a first policy for SharePoint applications, a second policy for blog applications, etc.), or other factors.

[0130] Figure 24 A conceptual illustration shows a set of application association policies 2400 for VHC according to some embodiments. In this case, application association policies 2400 include two different policies 2405 and 2410 for two different types of applications. As shown, each of these application association policies specifies instructions for generating and configuring logical forwarding elements to which the logical network endpoint (DCN) implementing the application is connected.

[0131] Application association policy 2405 (for Type 1 applications) specifies the creation of separate segments (e.g., logical switches) for each layer of the application and the assignment of subnets to each of these segments using the range of IP addresses available to the VHC. Specifically, the VHC will be assigned a set of private IP addresses (e.g., assigned by the tenant user who created the VHC or the user who created the VHC) for use by the DCN within the VHC to communicate internally with the virtualization infrastructure. The application association policy specifies the selection of subnets from this set of IP addresses to associate with the logical switch; DCNs added to the logical switch will be assigned IP addresses within that subnet. Application association policy 2405 also specifies the creation of T1 logical routers, to which each newly created segment is attached. Assuming at least one edge cluster in the physical infrastructure is already associated with the VHC, application association policy 2405 specifies the selection of this edge cluster to host one or more SRs for the T1 logical router. Finally, application association policy 2405 specifies the connection of the T1 logical router to a T0 logical router or T0VRF associated with the VHC in which the application is deployed (e.g., a provider network).

[0132] The second application association policy 2410 (for Type 2 applications, which in this example do not require external connectivity and may also have other specific characteristics) specifies the creation of a single fragment for all layers of the application and assigns a subnet to that fragment using the IP address range available to the VHC. That is, for these applications, all DCNs implementing any application layer are connected to the same logical switch. In this case, some embodiments use a computed expression associated with the DCN to distinguish the layers, so that the DCN can still be easily assigned to different security groups for different layers. For example, different embodiments allow the compute manager to assign tags or other metadata to the DCNs, indicating which layer each DCN belongs to. Other embodiments use specific IP addresses within the logical switch subnet to distinguish layers. As shown, the T1 logical router is still created according to the second application association policy 2410. Furthermore, the logical switch is still connected to this T1 logical router, and the edge cluster associated with the VHC is selected to host one or more SRs for the T1 logical router. In this case, the T1 logical router is not connected to any T0 logical router or VRF because external connectivity is presumed to be unnecessary. In some embodiments, this application association strategy 2410 can be used for applications during the testing phase.

[0133] Return to Figure 23 In process 2300 (at 2315), one or more segments (logical switches) are created for the application layer and IP ranges are assigned to these segments. In some embodiments, depending on the application association policy identified for the application definition, a logical switch can be defined for each layer, or a single logical switch can be defined for the application as a whole (which is shared between layers). As mentioned, if a single logical switch is defined, some embodiments use a computed expression associated with the DCN to differentiate layers, so that the DCN can still be easily assigned to different security groups for different layers.

[0134] Procedure 2300 also creates (at 2320) T1 logical routers to connect one or more segments created for the application layer and assigns one or more edge clusters to the T1 gateway (i.e., T1 SR). It should be noted that if centralized service is not required for the T1 logical routers (e.g., if the sole purpose of the T1 logical routers is to connect logical switches), then the T1 logical routers can be fully distributed without assigning edge clusters.

[0135] Process 2300 then determines (at 2325) whether connectivity to an external source is required. It should be noted that process 2300 is a conceptual process, and in some embodiments, this determination is part of a determination regarding the appropriate application of an association strategy. That is, whether connectivity is required determines the application of an association strategy, and the logical networking structure to be generated is simply specified by this strategy.

[0136] When an application (e.g., a specific layer of the application) requires external connectivity, process 2300 (at 2330) connects a T1 logical router created for the application to a T0 logical router or a T0 VRF (which is assigned to the VHC in which the application will be deployed). Whether the T0 logical router or one of the multiple VRFs associated with the T0 logical router is assigned to the VHC depends on which entities the provider exposes to the tenant (and, if the VHC is not directly managed by the tenant user, then it depends on which entities the tenant assigns to the sub-users managing the VHC).

[0137] Figure 25 A conceptual illustration shows an example set of logical network elements defined for application definition 2505 in policy tree 2500. As shown, application definition 2505 specifies web layer 2510, application layer 2515, and database layer 2520. This application corresponds to type 1 application (i.e., application-associated policy 2405), which can be explicitly indicated or determined based on application attributes.

[0138] Based on application association policy 2405, the network management system defines a set of logical network structures, which are shown in policy tree 2500. It should be understood that this policy tree is incomplete because it does not show either (i) security constructs defined for the application or (ii) any other logical networking constructs defined within the VHC (e.g., for other applications). Below the root node 2525 of policy tree 2500, policy tree 2500 includes a T1 logical router node 2530 and three logical switch nodes 2535-2545, each logical switch corresponding to a different layer of the application. Furthermore, some embodiments define a DHCP service 2550 associated with the T1 logical router for the application. This DHCP service (e.g., at an edge cluster associated with the T1 logical router 2530) operates to assign IP addresses to the DCN implementing the application layer connected to any of the logical switches 2535-2545. Finally, the T1 logical router connects to the T0 logical router represented by node 2555, which is part of the provider network but is exposed to the VHC represented by policy tree 2500.

[0139] Return to Figure 23In process 2300 (at 2335), managed network elements within the virtualization infrastructure are configured to implement a logical networking construct. As previously described, logical forwarding elements are implemented by the internal physical infrastructure of some embodiments (e.g., managed forwarding elements, such as virtual switches and virtual routers executed in the virtualization software of the host computer), and the network management and control system configures these elements to handle data traffic to and from the application DCN according to the newly defined logical forwarding elements. Similarly, edge devices implementing SRs for T1 logical routers (if such an SR is created) and T0 logical routers (if the T1 logical router is connected to a T0 logical router or a T0 VRF) are also configured to implement logical forwarding elements in their data paths.

[0140] Figure 26 Figure 2600 is conceptually illustrated, showing the overall networking and security implementation of a collection of applications created within a VHC. Figure 2600 illustrates different users managing different aspects of the VHC and applications, as well as security groups / policies and logical networking constructs defined for one of the applications. As shown, a DMZ security zone 2605 is created within a provider space governed by a provider administrator user (i.e., the highest-level user managing the network and control systems of the virtualization infrastructure). In some embodiments, the provider security administrator defines this security zone 2605, which can be used not only by VHC 2610 shown in the figure, but also by other VHCs of the same or other tenants. In this case, DMZ security zone 2605 is restricted to sites 1-3 by the provider security administrator.

[0141] VHC 2610 is managed by a VHC administrator (e.g., a tenant, a tenant's sub-users, etc.) and is defined to span all sites 1-4. In addition to DMZ security zone 2605 associated with the VHC, a VHC security administrator (which can be the same as the VHC administrator or a specific security administrator user created by the VHC administrator) defines production security zone 2615, which is restricted to sites 2-4. Unlike DMZ security zone 2605, production security zone 2615 is specific to VHC 2610.

[0142] Within the VHC, two distinct applications, 2620 and 2625, have been created by two different application owners. Application 2620 is a three-tier application with a web layer (assigned to DMZ security zone 2605, as shown by the dashed border), an application layer, and a database layer (both assigned to production security zone 2615, as shown by the long dashed border). Three workloads (e.g., VMs, containers, etc.) implement the web layer, and load balancing is used to distribute traffic among these workloads. Two workloads implement the application layer, and one workload implements the database layer. Application 2625 is a two-tier blog application with a web layer (assigned to DMZ security zone 2605) and a backend layer (assigned to production security zone 2615). Three workloads implement the web layer, and load balancing is again used to distribute traffic among these workloads, while a single workload implements the backend layer.

[0143] The bottom of Figure 2600 shows the security implementation 2630 and connectivity implementation 2635 for the three-tier application 2620. In this example, security groups are defined for the following: (i) the application's database layer, (ii) the application's application layer, (iii) the application's web layer, (iv) the application itself (including groups of all workloads in any application layer), (v) the DMZ security zone (i.e., any workload assigned to this security zone), and (vi) the production security zone. The first four groups are only relevant to this application and will create corresponding security groups for the second application 2625 (not shown), while the latter two groups corresponding to the security zones apply to any other applications deployed in this VHC.

[0144] In addition, the security policy is broken down into different segments, shown in order of priority. The highest priority segment is the DMZ segment (which contains rules for directing each application to an application-specific segment for data traffic between external endpoints and the web layer workloads used by those applications), followed by the production segment, and then the two application-specific segments of the security policy.

[0145] Connectivity implementation 2635 illustrates a logical network configured for a Layer 3 application 2620. In this configuration, three workloads at the web layer are attached to a first segment (logical switch), two workloads at the application layer are attached to a second segment, and a single workload at the database layer is attached to a third segment. Each of these segments is connected to a T1 logical router defined for the Layer 3 application, which in turn is connected to a T0 logical router (potentially managed by the provider).

[0146] at last, Figure 27The diagram conceptually illustrates at least a portion of a data model 2700 for a network management and control system according to some embodiments. This data model 2700 defines how logical networking and security constructs, as well as security zones and applications, are defined within the network management and control system. As shown, under the root, there are both (i) a provider segment 2705 and (ii) a tenant segment 2710. The provider segment 2705, managed by one or more provider users, allows providers to define (under the root node) logical networking constructs, including T0 logical routers and T0 VRFs belonging to these logical routers. For security aspects, provider users can define domains containing security policies (with one or more rules) and security groups. Furthermore, some embodiments allow provider users to define services (e.g., Layer 7 services) that security rules can reference.

[0147] The provider user also creates tenant user 2715, which manages tenant segment 2710 and specifies the physical sites that the tenant can use. It should be understood that the provider user can create any number of tenant users, each managing its own segment. A tenant can specify any number of sub-users, in this case, organizational units. The tenant, or these organizational units, can specify account sub-users (e.g., account 2720), and the creator of this account specifies that the account can be used for physical sites in one or more of its VHCs. Tenant users can also specify one or more shared security zones (e.g., DMZs) (these security zones are shared among all organizational units and / or accounts created under the tenant) and application service templates for connectivity, load balancing, or other services. As mentioned above, application service templates define the configuration of services consumed by the application layer (possibly defined based on details from the application developer), such as external connectivity, communication with backup servers, load balancing, IDS / IPS, etc.

[0148] Account user 2720 can also define VHC 2725, which can be configured by account user 2720 or a separate VHC administrator. Within VHC 2725, (in addition to shared security zones defined at the tenant level) account user 2720 or the VHC administrator can define security zones and restrict them to certain sites (provided those sites are within the VHC's scope). Account user 2720 or the VHC administrator can also define application association policies for the VHC, which specify how application definitions are translated into logical networking constructs.

[0149] Application owners (application developers) govern their application definitions 2730, which fall within the scope of VHC 2725. The application owner specifies the application, indicates the sites where the application can be deployed, and one or more application tiers. For each application tier, application services can be invoked for consumption by that tier by referencing an application service template defined by the tenant. Each application tier is automatically associated with a security zone, which is either defined within the VHC (as shown in this case) or shared between VHCs. Furthermore, application association policies are used to define and generate logical network structures (e.g., T1 logical routers, fragments) for the VHC policy tree under the root node. This policy tree also includes IP assignment information (IP pools and IP blocks) and security information (domains, security policies with rules, and security groups), which are specified by the VHC administrator (or other higher-level users) or automatically generated based on the application definitions used for the VHC. Security policy rules in the VHC can reference services and / or security groups defined within the VHC and / or within the provider policy tree. Similarly, security rules within the provider network can reference services and / or security groups defined within the provider policy tree and / or the VHC. These rules can designate security groups as the source and / or destination of traffic and indicate whether such traffic should be allowed.

[0150] Figure 28 An electronic system 2800, conceptually illustrated, is shown that implements some embodiments of the present invention. The electronic system 2800 may be a computer (e.g., a desktop computer, personal computer, tablet computer, server computer, mainframe, blade computer, etc.), a telephone, a PDA, or any other type of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 2800 includes a bus 2805, one or more processing units 2810, system memory 2825, read-only memory 2830, permanent storage device 2835, input device 2840, and output device 2845.

[0151] Bus 2805 broadly represents all system, peripheral, and chipset buses that communicatively connect numerous internal devices of electronic system 2800. For example, bus 2805 communicatively connects processing unit(s) 2810 to read-only memory 2830, system memory 2825, and permanent storage device 2835.

[0152] From these various memory units, processing unit(s) 2810 retrieves instructions to be executed and data to be processed in order to perform the process of the present invention. In different embodiments, the processing unit(s) may be a single processor or a multi-core processor.

[0153] Read-only memory (ROM) 2830 stores static data and instructions required by processing unit 2810 and other modules of the electronic system. On the other hand, permanent storage device 2835 is a read-write memory device. This device 2835 is a non-volatile memory unit that can store instructions and data even when the electronic system 2800 is turned off. Some embodiments of the invention use mass storage devices (such as magnetic disks or optical disks and their corresponding disk drives) as permanent storage device 2835.

[0154] Other embodiments use a removable storage device (such as a floppy disk, flash drive, etc.) as permanent storage device 2835. Like permanent storage device 2835, system memory 2825 is a read-write memory device. However, unlike storage device 2835, system memory is volatile read-write memory, such as random access memory. System memory stores some of the instructions and data required by the processor during runtime. In some embodiments, the processes of the present invention are stored in system memory 2825, permanent storage device 2835, and / or read-only memory 2830. From these various memory units, processing unit(s) 2810 retrieves instructions to be executed and data to be processed in order to perform the processes of some embodiments.

[0155] Bus 2805 is also connected to input and output devices 2840 and 2845. Input devices enable users to communicate information and select commands to the electronic system. Input device 2840 includes an alphanumeric keypad and a pointing device (also known as a "cursor control device"). Output device 2845 displays images generated by the electronic system. Output devices include printers and display devices such as cathode ray tubes (CRTs) or liquid crystal displays (LCDs). Some embodiments include devices that serve as both input and output devices, such as touchscreens.

[0156] Finally, as Figure 28 As shown, bus 2805 also couples electronic system 2800 to network 2865 via a network adapter (not shown). In this way, the computer can be part of a computer network (such as a local area network (“LAN”), wide area network (“WAN”), or intranet) or a network of networks (such as the Internet). Any or all components of electronic system 2800 can be used in conjunction with this invention.

[0157] Some embodiments include electronic components, such as microprocessors, storage devices, and memories, that store computer program instructions in machine-readable or computer-readable media (which may alternatively be referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Examples of such computer-readable media include RAM, ROM, read-only optical discs (CD-ROM), recordable optical discs (CD-R), rewritable optical discs (CD-RW), read-only digital multifunction discs (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and / or solid-state hard disk drives, read-only and recordable... Disks, high-density optical discs, any other optical or magnetic media, and floppy disks. Computer-readable media can store computer programs that can be executed by at least one processing unit and include a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as machine code generated by a compiler, and files that include high-level code executed by a computer, electronic component, or microprocessor using an interpreter.

[0158] While the above discussion primarily concerns microprocessors or multi-core processors that execute software, some embodiments are executed by one or more integrated circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuit itself.

[0159] As used in this specification, the terms "computer," "server," "processor," and "memory" all refer to electronic or other technical devices. These terms do not include people or groups of people. For the purposes of this specification, the term "display" refers to display on an electronic device. As used in this specification, the terms "computer-readable medium" and "machine-readable medium" are entirely limited to tangible physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, or any other transient signals.

[0160] This specification deals with computing and networking environments including virtual machines (VMs) throughout. However, a virtual machine is just one example of a data compute node (DCN) or data compute endpoint (also known as an addressable node). A DCN may include non-virtualized physical hosts, virtual machines, containers that run on top of a host operating system without a hypervisor or a separate operating system, and hypervisor kernel network interface modules.

[0161] In some embodiments, VMs operate on the host machine alongside their own guest operating system, using resources virtualized by virtualization software (e.g., hypervisors, virtual machine monitors, etc.). Tenants (i.e., the VM owners) can choose which applications run on top of the guest operating system. On the other hand, some containers are constructed to run on top of the host operating system without requiring a hypervisor or a separate guest operating system. In some embodiments, the host operating system uses namespaces to isolate containers from each other, thus providing operating system-level isolation for different groups of applications operating within different containers. This isolation is similar to the VM isolation provided in a hypervisor virtualization environment of virtualization system hardware and can therefore be viewed as a form of virtualization that isolates different groups of applications operating in different containers. Such containers are more lightweight than VMs.

[0162] In some embodiments, the hypervisor kernel network interface module is a non-VM DCN that includes a network stack with a hypervisor kernel network interface and receive / transmit threads. An example of a hypervisor kernel network interface module is the vmknic module, which is used by VMware's ESXi. TM Part of the management process.

[0163] It should be understood that although the specification mentions VMs, the examples given can be of any type of DCN, including physical hosts, VMs, non-VM containers, and hypervisor kernel network interface modules. In fact, in some embodiments, the example network may include a combination of different types of DCNs.

[0164] Although the invention has been described with reference to many specific details, those skilled in the art will recognize that the invention can be embodied in other specific forms without departing from its spirit. Furthermore, several figures (including...) Figure 12 , 16 (20) and (23) conceptually illustrate the process. The specific operations of these processes may not be performed in the exact order shown and described. Specific operations may not be performed in a continuous sequence of operations, and different specific operations may be performed in different embodiments. Furthermore, the process may be implemented using several sub-processes, or as part of a larger macro-process. Therefore, those skilled in the art will understand that the invention is not limited to the foregoing illustrative details, but is defined by the appended claims.

Claims

1. A method for a network management and control system for managing one or more logical networks, the method comprising: The provider first user of the network management and control system receives the definition of a first set of one or more security zones for a logical network, each security zone definition including a set of security rules for assigning to the data computing node DCN of the security zone, wherein the provider first user defines multiple tenant users of the network management and control system. The second tenant user receives the definition of a second set of security zones of the logical network from the second tenant user of the network management and control system, the second tenant user being one of the plurality of tenant users defined by the first provider user, and manages the networking and security configuration of at least a portion of the logical network; The application definition to be deployed in the logical network is received from the application developer, a third user, of the network management and control system. The application definition specifies a set of security requirements for the application. as well as Based on the specified set of security requirements, the DCN implementing the application is assigned to one or more security zones in the first set and the second set of security zones for the logical network.

2. The method of claim 1, wherein: The first security zone includes rules that allow connections between the DCN within the first security zone and endpoints outside the logical network; and The second security zone does not allow connections between the DCN within the second security zone and endpoints outside the logical network.

3. The method as described in claim 2, wherein, Only DCNs assigned to the first security zone are allowed to connect to endpoints outside the logical network.

4. The method of claim 2, wherein, The second security zone includes rules that allow connections between DCNs in the first security zone and DCNs in the second security zone.

5. The method of claim 1, wherein, The logical network spans at least two virtual clouds, wherein a DCN in the first virtual cloud is not allowed to communicate with a DCN in the second virtual cloud until the DCN is assigned to a specific security zone.

6. The method of claim 5, wherein, The first DCN in the first virtual cloud assigned to the first security zone communicates with the second DCN in the second virtual cloud assigned to the second security zone.

7. The method of claim 6, wherein, The first DCN belongs to a specific layer of the application, while the second DCN does not belong to any application.

8. The method of claim 1, wherein, The definition of the application includes at least two application layers.

9. The method of claim 8, wherein, DCNs belonging to the first application layer are assigned to the first security zone, and DCNs belonging to the second application layer are assigned to the second security zone.

10. The method of claim 9, wherein: The first application layer is the web layer that receives communication from external devices, and the second application layer is the database layer that is restricted from receiving communication from external devices. DCNs assigned to the first security zone are allowed to receive ingress connections from Internet sources, while DCNs assigned to the second security zone are only allowed to receive ingress connections from DCNs assigned to the first security zone.

11. The method of claim 1, wherein, The set of security rules defining the security zone is implemented through firewall rules applied by network elements managed by the network management and control system.

12. The method of claim 1, wherein the application developer, a third user, is not authorized to access the networking and security configurations used for the logical network.

13. The method of claim 1, wherein the first set of security zones is defined for multiple logical networks managed by the plurality of tenant users, the plurality of tenant users including the second tenant user.

14. A machine-readable medium storing a program, said program, when implemented by at least one processing unit, implementing the method as described in any one of claims 1-13.

15. An electronic device comprising: A collection of processing units; as well as A machine-readable medium storing a program that, when implemented by at least one of the processing units, implements the method as described in any one of claims 1-13.

16. A system comprising components for implementing the method as claimed in any one of claims 1-13.

17. A computer program product comprising instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-13.

Images