Distributed network plug-in agent for container networking

Abstract

Some embodiments of the present invention provide a method for connecting a group of one or more deployed machines in a software-defined data center (SDDC) to a virtual private cloud (VPC) in an availability zone (AZ). The method deploys network plug-in agents (e.g., listening agents) on multiple host computers and configures the network plug-in agents to receive notifications of events relating to the deployment of network elements from a set of compute deployment agents that execute on the host computers of the specific deployed network plug-in agents. In some embodiments, the method is performed by a network manager that receives notifications of events relating to the deployed machines from the deployed network plug-in agents and, in response to the received notifications, configures network elements to connect one or more groups of deployed machines.

Description

Background Technology

[0001] With the recent surge in cloud-native applications, the need for rapid, on-demand networking of machines deployed in software-defined data centers (SDDCs) is greater than ever. As the number of compute clusters increases, each interacts independently with the SDDC's network manager, which tends to be overwhelmed by requests from the compute clusters for connecting to machines deployed in the SDDC. Therefore, a solution is desired that allows compute clusters to communicate with the SDDC's network manager without overwhelming it. Summary of the Invention

[0002] Some embodiments of the present invention provide a method for connecting a group of one or more deployed machines in a software-defined data center (SDDC) to a virtual private cloud (VPC) in an availability zone (AZ). The method deploys network plug-in agents (e.g., listening plug-ins) on multiple host computers and configures these network plug-in agents to receive notifications of events related to the deployment of network elements from a set of compute deployment agents, which execute on the host computers of the specific deployed network plug-in agents. In some embodiments, the method is performed by a network manager that receives notifications of events related to the deployed machines from the deployed network plug-in agents and, in response to the received notifications, configures network elements to connect to one or more groups of deployed machines. In some embodiments, the network manager is the network manager of the VPC in the AZ.

[0003] In some embodiments, configuring a network plug-in agent to receive notifications includes registering the network plug-in agent with an API (Application Programming Interface) processor that receives intent-based API requests and parses these requests to identify: (1) a group of machines to be deployed and / or modified in a machine group, (2) a group of network elements to connect to the group of machines, or (3) a group of service machines to perform services for the group of machines. In some embodiments, the API is a hierarchical document that can specify multiple different compute and / or network elements at different levels of a compute and / or network element hierarchy.

[0004] In some embodiments, API requests include requests to add, remove, or modify deployed machines. In some embodiments, a machine is a container or pod deployed in Kubernetes. In some embodiments, a machine is connected to a VPC by a network manager by assigning the machine a set of network addresses (e.g., IP (Internet Protocol) address, MAC (Media Access Control) address, port, etc.) and updating a set of network elements (e.g., forwarding elements) of the VPC to connect to the machine using at least one of the network addresses in the set.

[0005] In some embodiments, the network plug-in agent registers with compute deployment agents running on multiple host computers to receive notifications via a hyperbus. In some embodiments where the network plug-in agent receives notifications from compute deployment agents on other host computers, the hyperbus is a logical network spanning multiple host computers dedicated to exchanging data messages (i.e., registration and notification messages) between the network plug-in agent and the compute deployment agent. In some embodiments, the compute deployment agent runs on a set of master worker nodes (e.g., virtual machines) within a corresponding Kubernetes worker node cluster (e.g., a compute cluster including master nodes and worker nodes), which has worker nodes running on multiple host computers. In some embodiments, the set of master worker nodes monitors other worker nodes in the Kubernetes worker node cluster to detect failures in components of the Kubernetes worker node cluster. When a master worker node detects a failure, in some embodiments, the master worker node generates a request to deploy a replacement for the failed component.

[0006] In some embodiments, the network plug-in agent receives notifications from a group of worker nodes running on the same host computer via a hyperbus on the host computer. In some embodiments, each worker node executes a notification agent to notify the network plug-in agent of events related to the deployment of machines on that worker node, thus limiting the hyperbus to the host computer. In some embodiments, the notification agent forwards CRUD (create, read, update, and delete) requests received at the worker nodes for deploying machines (e.g., containers, pods, applications, etc.) to the network plug-in agent. In some embodiments where a notification agent is executed on each worker node, the hyperbus is a logical network spanning only a single host computer, connecting the network plug-in agent and worker nodes running on that host computer.

[0007] In some embodiments, the network plug-in agent is an instance of a distributed network plug-in agent (DNPA). In some embodiments, the DNPA includes (1) a listening plug-in that receives notifications from a compute deployment agent, (2) a request queue for storing requests related to notifications received by the network plug-in agent, (3) a cluster manager plug-in (CMP) that processes requests related to cluster-level requests stored in the request queue, (4) a pod manager plug-in (PMP) that processes requests related to pod-level requests related to worker nodes executing on a host computer stored in the request queue, and (5) a communication agent that forwards the requests processed by the cluster manager plug-in and the pod manager plug-in to a network manager, wherein the network manager configures network elements in the availability zone based on the requests processed by the cluster manager and the pod manager.

[0008] In some embodiments, a specific DNPA instance is selected as the primary DNPA instance. To ensure that the network manager is not overwhelmed by requests from multiple DNPA instances, the primary DNPA instance receives data on the number of pending requests stored in a request queue, which are associated with the DNPA instance executing on each of the multiple host computers. The primary DNPA instance determines the total number of pending requests related to the machine deployment and the maximum rate at which requests can be processed at the network manager. Based on the data on the number of pending requests and the determined maximum rate at which requests can be processed at the network manager, the primary DNPA instance calculates a rate limit for each DNPA instance executing on the multiple host computers. This rate limit specifies the rate at which the DNPA instance is allowed to send requests related to the machine deployment to the network manager. In some embodiments, a rate limit calculator is implemented in the primary DNPA instance. The primary DNPA then distributes the rate limit calculated for each DNPA instance to the DNPA instance for which its rate limit is calculated.

[0009] In some embodiments, the primary DNPA instance selects a cluster manager plugin for a specific DNPA instance as the primary cluster manager plugin for each of a set of worker node clusters (e.g., a compute cluster). In some embodiments, this primary cluster manager plugin is the only cluster manager plugin that communicates with the network manager regarding events relating to these clusters (for which the cluster manager is selected as the primary cluster manager plugin). In some embodiments, the primary DNPA instance receives data about the resources available on each of the multiple host computers, and the primary cluster manager plugin for each worker node cluster is selected based on the received data about the resources available on each of the multiple host computers.

[0010] In some embodiments, each DNPA instance receives a notification and determines whether the notification relates to: (1) a cluster-level request related to a cluster for which a cluster manager plugin on the same host computer has been designated as the primary cluster manager plugin, or (2) a pod-level request related to a worker node running on the same host computer. For each notification determined to be related to either of the following, the DNPA instance adds the request related to the received notification to a request queue: 1) a cluster-level request related to a cluster for which a cluster manager plugin on the same host computer has been designated as the primary cluster manager plugin, or (2) a pod-level request related to a worker node running on the same host computer.

[0011] In some embodiments, the received notification is a set of intent-based API requests forwarded from an API processor executing on the master network node. This set of received intent-based API requests is parsed by DNPA to generate at least one of a first set of cluster-level requests and a second set of pod-level requests, identifying at least one of the cluster-level and pod-level requests, for storage in a request queue on the host computer. In some embodiments, these requests added to the request queue are identified as either cluster-level or pod-level requests. In some embodiments, the number of cluster-level requests and the number of pod-level requests are reported separately to the master DNPA instance. In some embodiments, the master DNPA instance calculates separate rate limits for cluster-level and pod-level requests for each DNPA instance, and these rate limits are distributed to the DNPA instances.

[0012] The preceding summary section is intended as a brief introduction to some embodiments of the invention. It is not intended to be an introduction or overview of all inventive subjects disclosed in this document. The following detailed description, along with the accompanying drawings referenced in the detailed description, will further describe the embodiments described in the summary and other embodiments. Therefore, a full examination of the summary, detailed description, drawings, and claims is necessary to understand all the embodiments described in this document. Furthermore, the claimed subject matter is not limited to the illustrative details in the summary, detailed description, and drawings, but is defined by the appended claims, as the claimed subject matter may be embodied in other specific forms without departing from the spirit of the subject matter. Attached Figure Description

[0013] The novel features of the invention are set forth in the appended claims. However, for illustrative purposes, several embodiments of the invention are illustrated in the following drawings.

[0014] Figure 1 Examples of control systems according to some embodiments of the present invention are illustrated.

[0015] Figure 2 The illustration shows an example of a logical network for defining a VPC for an entity, such as a company in a multi-tenant public data center or a division of a company in a private data center.

[0016] Figure 3 The concept illustrates the process of deploying an NPA on a set of host computers on a node hosting a VPC.

[0017] Figure 4 The diagram illustrates the process based on... Figure 3 The host computer system is configured after the process.

[0018] Figure 5 The concept illustrates the process by which the DNPA controls the rate at which it provides notifications to the network manager.

[0019] Figure 6 The illustration shows an example of a host computer implementing a single DNPA to select a DNPA master instance for a cluster of different entities.

[0020] Figure 7 The concept illustrates the process by which DNPA performs the selection of a primary CMP for a cluster of deployed worker nodes associated with DNPA and processes cluster-level requests at the selected primary CMP.

[0021] Figure 8 The concept illustrates the process of parsing and queuing received notifications at a DNPA instance.

[0022] Figure 9 The concept illustrates the process for handling queued requests at a DNPA instance.

[0023] Figure 10A-10D The diagram illustrates a set of operations performed on two clusters spanning three nodes, with each node executing a DNPA instance.

[0024] Figure 11 The illustration shows an example of a distributed load balancer in some embodiments, which is defined by the control system for several VIF-associated machines on several host computers.

[0025] Figure 12 The illustration shows an example of a distributed firewall in some embodiments, which is defined by the control system for several VIF-associated machines on several host computers.

[0026] Figure 13 A computer system that implements some embodiments of the present invention is conceptually illustrated. Detailed Implementation

[0027] In the following detailed description of the invention, numerous details, examples, and embodiments of the invention are set forth and described. However, it will be clear and apparent to those skilled in the art that the invention is not limited to the described embodiments, and that the invention can be practiced without discussing some specific details and examples.

[0028] Some embodiments of the present invention provide a method for connecting a group of one or more deployed machines in a software-defined data center (SDDC) to a virtual private cloud (VPC) in an availability zone (AZ). The method deploys network plug-in agents (e.g., listening plug-ins) on multiple host computers and configures these agents to receive notifications of events relating to the deployment of network elements from a set of compute deployment agents that execute on the host computers of the specific deployed network plug-in agents. In some embodiments, the method is performed by a network manager that receives notifications of events relating to the deployed machines from the deployed network plug-in agents and, in response to the received notifications, configures network elements to connect to one or more groups of deployed machines. In some embodiments, the network manager is the network manager of the VPC in the AZ.

[0029] In some embodiments, configuring a network plug-in agent to receive notifications includes registering the network plug-in agent with an API (Application Programming Interface) processor that receives intent-based API requests and parses these requests to identify (1) a group of machines to be deployed and / or modified in a machine group, (2) a group of network elements to connect to that group of machines, or (3) a group of service machines to perform services for that group of machines. In some embodiments, the API is a hierarchical document that may specify multiple different computing and / or network elements at different levels of a computing and / or network element hierarchy.

[0030] In some embodiments, the API defines a cluster of nodes (e.g., a Kubernetes worker node cluster), which includes a set of components representing the control plane of the cluster and a set of (worker) nodes. In some embodiments, a node is a host computer that hosts components of the Kubernetes cluster. In some embodiments, the host computer of the cluster is a physical machine, a virtual machine, or a combination of both. The host computer (i.e., the node) executes a set of Pods, which in some embodiments include a set of containers. In some embodiments, the Kubernetes worker nodes execute a proxy (e.g., a kubelet) that ensures containers run within Pods, a container runtime that is responsible for running the containers, and a network proxy (e.g., kube-proxy). In some embodiments, the cluster is divided into a set of namespaces, and different Pods or containers are deployed to this set of namespaces.

[0031] In some embodiments, the network control system performs an automated process to define a Virtual PC (VPC) to connect a group of machines to a logical network that isolates these machines from other machines in the data center. In some embodiments, the group of machines includes virtual machines (VMs) and container pods, the VPC is defined using a supervisor cluster namespace, and API requests are provided as YAML files. In some embodiments, the deployed logical network is a Kubernetes-based logical network that defines VPCs for corporate entities in one or more data centers. In some embodiments, the VPC is a “supervisor” Kubernetes cluster with a namespace that provides a tenancy boundary for the entity. These embodiments use custom resource definitions (CRDs) to define additional network constructs and policies that complement native Kubernetes resources. The network control system is further described in detail in U.S. Patent Application No. 16 / 897,652, filed June 10, 2020, which is incorporated herein by reference.

[0032] In some embodiments, a VPC's logical network interconnects a set of deployed machines. For example, in some embodiments, the logical network includes one or more logical forwarding elements, such as logical switches, routers, gateways, etc. In some embodiments, this method defines logical forwarding elements (LFEs) by configuring several physical forwarding elements (PFEs), some or all of which execute on a host computer along with the deployed machines (e.g., VMs and Pods). In some embodiments, a PFE is configured to implement two or more LFEs to connect two or more distinct subsets of the deployed machines.

[0033] Those skilled in the art will recognize that other embodiments define different types of networks for other types of entities, such as other business entities, non-profit organizations, educational entities, etc. In some of these other embodiments, neither Kubernetes nor Kubernetes-based Pods are used. For example, some embodiments are used to deploy networks solely for VMs and / or non-Kubernetes containers / Pods.

[0034] As used in this document, a data message refers to a specific set of bits transmitted across a network. Those skilled in the art will recognize that, as used in this document, the term data message refers to various formatted sets of bits transmitted across a network. The formatting of these bits can be specified by standardized or non-standardized protocols. Examples of data messages conforming to standardized protocols include Ethernet frames, IP packets, TCP segments, UDP datagrams, etc. Furthermore, as used in this document, references to layers L2, L3, L4, and L7 (or layers 2, 3, 4, and 7) refer to the second data link layer, third network layer, fourth transport layer, and seventh application layer of the OSI (Open Systems Interconnection) layer model, respectively.

[0035] Figure 1 An example of a control system 100 according to some embodiments of the present invention is illustrated. This system 100 processes APIs that use a Kubernetes-based declarative model to describe the desired state of: (1) the machines to be deployed, and (2) the connectivity, security, and service operations to be performed for the deployed machines (e.g., private and public IP address connectivity, load balancing, security policies, etc.). To process these APIs, the control system 100 uses one or more CRDs that define the attributes of custom-specified network resources referenced by the received API requests. The system 100 performs an automated process to deploy a logical network that connects the deployed machines and isolates them from other machines in the data center. In some embodiments, the machines are connected to the deployed logical network of a VPC.

[0036] As shown in the figure, the control system 100 includes an API processing cluster 105, a software-defined networking (SDN) manager cluster 110, an SDN controller cluster 115, and a compute manager and controller 117. The API processing cluster 105 includes two or more API processing nodes 135, each node including an API processing server 140. The API processing server 140 (e.g., a compute deployment agent) communicates directly with a network plug-in agent (NPA) 145 executing on a host computer 150, or indirectly with the NPA 145 through an agent executing on a Kubernetes node 135. In some embodiments, the NPA 145 is part of a distributed NPA 155. The API processing server 140 receives and parses intent-based API calls. In some embodiments, the received API calls are in a declarative, layered Kubernetes format and may contain multiple different requests. In some embodiments, the API calls are received from a system administrator, a cluster user, or generated by an automation script.

[0037] API processing server 140 parses each received intent-based API request into one or more individual requests. When the request relates to machine deployment, API server 140 provides these requests directly to compute manager and controller 117, or indirectly to compute manager and controller 117 via an agent running on Kubernetes master node 135. Compute manager and controller 117 then deploy VMs (e.g., Kubernetes worker nodes) and / or Pods on host machines in the Availability Zone.

[0038] API calls may also include requests to deploy network elements. In some embodiments, these requests explicitly identify the network elements to be deployed, while in other embodiments, they may implicitly identify these network elements by requesting the deployment of a computing architecture (e.g., a compute cluster, container, etc.) for which network elements must be defined by default. As further described below, the control system 100 uses NPA 145 to identify network elements to be deployed and to guide the deployment of these network elements.

[0039] NPA 145 is the interface between API server 140 and SDN manager cluster 110, which manages network elements used as forwarding elements (e.g., switches, routers, bridges, etc.) and service elements (e.g., firewalls, load balancers, etc.) in the availability zone. SDN manager cluster 110 instructs SDN controller cluster 115 to configure these network elements to implement desired forwarding and / or service elements (e.g., logical forwarding elements and logical service elements) for one or more logical networks. As further described below, in some embodiments, SDN controller cluster 115 interacts with local controllers on edge gateways and host computers to configure network elements.

[0040] In some embodiments, NPA 145 registers event notifications with API server 140, for example, by establishing a long-pull session with API server 140 to receive all CRUD events defined for networking for various CRDs. In some embodiments, API server 140 is a Kubernetes master VM, and NPA 145 runs on host computer 150 on which the Kubernetes master VM executes. In some embodiments, NPA 145 collects and provides implementation data from SDN resources for CRDs, as it relates to the CRD state. In some embodiments, each Kubernetes master VM monitors other worker nodes in the Kubernetes worker node cluster to detect failures in components of the Kubernetes worker node cluster. In some embodiments, when the Kubernetes master VM detects a failure, the Kubernetes master VM generates a request to deploy a replacement for the failed component.

[0041] In some embodiments, NPA 145 processes parsed API requests related to deployed machines in the network to instruct SDN Manager Cluster 110 to implement: (1) Virtual Interfaces (VIFs) required to connect VMs and Pods to forwarding elements on host computers; (2) Virtual Networks to implement different segments of the logical network of the VPC; (3) Load Balancers to distribute traffic load to endpoint machines; (4) Firewalls to implement security and management policies; and (5) Exposed ports to access services provided by a group of machines in the VPC to machines outside and inside the VPC. Details of the implementation of these different components can be found in U.S. Patent Application No. 16 / 897,652.

[0042] In some embodiments, API server 140 provides CRDs already defined for the extended network construct to NPA 145, enabling it to process APIs referencing the corresponding network construct. API server 140 also provides configuration data from a configuration storage area to NPA 145. In some embodiments, this configuration data includes parameters adjusted by predefined template rules that NPA 145 follows to execute its automated processes. NPA 145 executes these automated processes to execute received API requests in order to guide SDN manager cluster 110 in deploying or configuring network elements for a VPC. For a received API, control system 100 executes one or more automated processes to identify and deploy one or more network elements for implementing the logical network of the VPC. After an API request is received, control system 100 executes these automated processes without any administrator action to guide the identification and deployment of network elements.

[0043] SDN Manager 110 and Controller 115 can be any SDN manager and controller available today. In some embodiments, these managers and controllers are NSX-T managers and controllers licensed by VMware, Inc. In such embodiments, NPA 145 detects network events by processing data supplied by its corresponding API server 140 and uses the NSX-T API to instruct NSX-T Manager 110 to deploy and / or modify the NSX-T network constructs required to realize the network state expressed by the API calls. Communication between NPA 145 and NSX-T Manager 110 is asynchronous, wherein NPA 145 provides the desired state to NSX-T Manager 110 and then relays the desired state to NSX-T Controller 115 to asynchronously compute the state and propagate it to host computers, forwarding elements, and service nodes in the Availability Zone (i.e., to the SDDC group controlled by Controller 115).

[0044] After receiving the API from NPA 145, in some embodiments, SDN manager 110 instructs SDN controller 115 to configure network elements to achieve the network state expressed by the API call. In some embodiments, SDN controller 115 serves as the central control plane (CCP) of control system 100. Figure 2 The SDN controller 115 is depicted as a CCP that computes advanced configuration data (port configurations, policies, forwarding tables, service tables, etc.). With this capability, the SDN controller 115 pushes the advanced configuration data to the Local Control Plane (LCP) agent 220 on the host computer 205, the LCP agent 225 on the edge device 210, and the TOR agent 230 on the top-of-rack (TOR) switch 215.

[0045] Based on the received configuration data, the LCP agent 220 on host 205 configures one or more software switches 250 and software routers 255 to implement distributed logical switches, routers, bridges, and / or service nodes (e.g., service VMs or monitoring engine) of one or more logical networks, along with corresponding switches and routers on other host computers 205, edge devices 210, and TOR switches 215. On edge device 210, the LCP agent 225 configures the packet processing phase 270 of these devices to implement logical switches, routers, bridges, and / or service nodes of one or more logical networks, along with corresponding switches and routers on other host computers 205, edge devices 210, and TOR switches 215.

[0046] For TOR switch 215, TOR agent 230 configures one or more configuration tables 275 of TOR switch 215 via OVSdb server 240. Data in configuration table 275 is then used to configure hardware ASIC packet processing pipeline 280 to perform desired forwarding operations to achieve desired logical switching, routing, bridging, and service operations. CCP, LCP, and TOR agent are described in more detail in U.S. Patent Nos. 10,554,484, 10,250,553, 9,847,938, and 9,178,833, and are incorporated herein by reference.

[0047] After the host computer 205 is configured together with the edge device 210 and / or the TOR switch 215, they can implement one or more logical networks, wherein each logical network isolates the machine and network traffic of the entity to which it is deployed from the machine and network traffic of other entities in the same availability zone. Figure 2 An example of a logical network 295 for defining a VPC for an entity (such as a company in a multi-tenant public data center, or a division of a company in a private data center).

[0048] As shown in the figure, the logical network 295 includes multiple logical switches 284, each logical switch 284 connecting different groups of machines and serving as different network segments. Each logical switch 284 has a port 252 connected to (i.e., associated with) a virtual interface 265 of a machine 260. In some embodiments, the machine 260 includes VMs and Pods, each Pod having one or more containers.

[0049] Logical network 295 also includes logical router 282 connecting different network segments defined by different logical switches 284. In some embodiments, logical router 282 is used as... Figure 2The gateway for the VPC deployed in the middleware. In some embodiments, the logical router 282 includes a distributed routing component 296 and a centralized routing component 297. In some embodiments, the distributed routing component 296 is implemented by routing instances running on the host computer 205 and the edge device 210, while the centralized routing component 297 is implemented by the edge device 210. Each centralized routing component 297 performs one or more services 291, or is associated with one or more middleware service nodes performing one or more services. Therefore, in some embodiments, the centralized routing component 297 is referred to as a service router.

[0050] In some embodiments, centralized and distributed routing components 296 and 297 are connected via a logical switch 294 defined on host computer 205 and edge device 210. Furthermore, in some embodiments, logical router 282 is implemented by a pair of logical nodes 299, each node having both centralized and distributed components. In some embodiments, the pair of nodes 299 may be configured to operate in active / active or active / standby modes. U.S. Patent No. 9,787,605 describes gateway implementations of some embodiments in more detail and is incorporated herein by reference.

[0051] Figure 3 A conceptual illustration of process 300 for deploying an NPA on a set of host computers of a node hosting a VPC is provided. In some embodiments, process 300 is performed by a set of network managers (e.g., SDN manager 110). The process begins by identifying (at 305) a set of host computers of the node (or other component) hosting the VPC. In some embodiments, this identification is performed as part of executing instructions received at the network manager for implementing the new VPC.

[0052] After identifying (at 305) the group of host computers hosting the VPC components, process 300 generates (at 310) configuration data for configuring the identified group of host computers to deploy an NPA to receive event notifications from a group of compute deployment agents (e.g., API server 140) executing on the host computers of a particular deployed NPA. In some embodiments, the configuration data includes specifications for a logical network (sometimes referred to as a hyperbus) for communication between the compute deployment agents and the NPA. In some embodiments, the hyperbus spans a single host computer, while in other embodiments, the hyperbus (e.g., a logical network) spans multiple host computers and enables communication between compute deployment agents executing on one host computer to communicate with the NPA deployed on each of the multiple host computers. In some embodiments, the configuration data includes data for registering the NPA with a group of compute deployment agents connected to the same logical network (e.g., the hyperbus) to initiate a long-connection operation to receive notifications of events related to the VPC. In some embodiments, registering the NPA is done by registering a listening agent of the NPA (such as below) Figure 4 This is accomplished by the listening plugin 485 described in the text.

[0053] The generated configuration data is then sent (at 315) to the network controller (e.g., SDN controller 115) to configure the host computer for NPA deployment. In some embodiments, the NPA is a distributed NPA (DNPA) implemented together as individual instances of that DNPA (e.g., DNPA 155). After the NPA is deployed, process 300 receives (at 320) notifications from at least one deployed NPA regarding events relating to at least one machine in the VPC. In some embodiments, the notification relates to at least one request for adding, removing, or modifying a machine in the VPC, requiring updates to the VPC's network elements. In some embodiments, the notification is a request to a network manager (e.g., SDN manager 110) to instruct the network manager to implement the following: (1) virtual interfaces (VIFs) required to connect VMs and Pods to forwarding elements on a host computer; (2) virtual networks used to implement different segments of the logical network of the VPC; (3) load balancers used to distribute traffic loads to endpoint machines; (4) firewalls used to implement security and management policies; and (5) exposed ports used to access services provided by a group of machines in the VPC to machines outside and inside the VPC.

[0054] Process 300 then generates (at 325) configuration data for configuring network elements based on the received notification. After generating (at 325) the configuration data, the generated configuration data is sent (at 330) to the network controller to configure the network elements of the VPC. Process 300 ends after the configuration data is sent to the network controller. Figure 4 The diagram illustrates system 400 after host computers 450a-n are configured according to process 300. A group of host computers 450a-n execute DNPA instances 445a-n, which together implement DNPA 455 in the AZ. Figure 4 The components will be combined to describe the process executed by DNPA. Figure 5 , 7 Let's discuss it with number 8.

[0055] Figure 5 A conceptual illustration of process 500 for DNPA to control the rate at which DNPA (i.e., DNPA instances implementing DNPA together) provides notifications (e.g., makes requests) to the network manager. In some embodiments, a single DNPA instance executes on each of multiple host computers, replacing multiple network agents that independently provide notifications to the network manager and execute on each host of a multiple cluster. In some embodiments, this rate limiting prevents the individual network agents from overwhelming the network manager and causing network manager failure. In some embodiments, process 500 is performed by DNPA.

[0056] After the DNPA instance is deployed as described with respect to process 300, process 500 begins by selecting (at 505) a primary DNPA instance. In some embodiments, the primary DNPA instance is selected based on the available resources on the host computer on which the DNPA instance is deployed. The selected primary DNPA instance (e.g., DNPA instance 445a) then receives (at 510) data regarding pending requests at each DNPA instance, including the primary DNPA instance itself. In some embodiments, this data is received via a specific network (e.g., a logical switch, VLAN, etc.) created for communication between DNPA instances. The primary DNPA aggregates the received data regarding pending requests to determine the total number of pending requests. In some embodiments, this data is aggregated by a rate limiting calculator (e.g., rate limiting calculator 481a) of the primary DNPA instance. In some embodiments, pending requests are identified as being associated with cluster-level events (e.g., events related to namespaces, logical networks within the cluster, load balancers, network policies, etc.) or pod-level requests. In some embodiments, communication agent 482a receives pending request data from pod manager plugin 483a and cluster manager plugin 484a, while in other embodiments, request queue 486a communicates directly with communication agent 482a to identify the number of pending requests.

[0057] In addition to receiving (at 510) data about pending requests, the DNPA also determines (at 515) the maximum rate at which it provides notifications to the network manager (e.g., SDN manager 410). In some embodiments, the maximum rate is configured when the DNPA is deployed and can be updated as network conditions change. In some embodiments, the maximum rate is based on the number of DNPAs deployed for different VPCs. Based on the number of pending requests associated with each DNPA instance and the maximum rate for providing notifications from the DNPA, the primary DNPA instance calculates (e.g., using a rate limit calculator 481a) (at 520) a rate limit 470 for communication with the network manager for each DNPA instance. In some embodiments, the rate limit calculator is inactive in all other DNPA instances.

[0058] In some embodiments, the primary DNPA instance calculates separate rate limits for cluster-level and pod-level notifications for each DNPA instance. In some embodiments, the rate limit for a particular DNPA instance is calculated based on the number of pending requests associated with that DNPA instance and the total number of pending requests. For example, the rate limit calculated for a particular DNPA would be the maximum rate allowed at the network manager multiplied by the number of pending requests associated with that DNPA instance and divided by the total number of pending requests associated with the DNPA (e.g., all DNPA instances).

[0059] The calculated rate limits 470a-n are then distributed (at 525) to each DNPA instance via communication agents (e.g., communication agents 482a-n) to limit the rate at which the DNPA notifies the network manager of events related to machines in a VPC that require configuration of network elements. The DNPA (e.g., an instance of the DNPA) then sends (at 530) a request to the network manager to update the network elements to take into account the event notifications received at the DNPA, subject to the calculated rate limits. In some embodiments, the communication agent of each DNPA instance enforces this rate limit when communicating with the SDN manager 410. In some embodiments, operations 510-530 are performed periodically to ensure that request queues are processed based on their current size rather than their historical size.

[0060] In some embodiments, DNPA supports clusters of different tenants or VPCs and receives notifications from compute deployment agents from nodes in different VPCs or belonging to different tenants, and communicates with different SDN manager clusters. Figure 6 The illustration depicts an embodiment where a host computer implements a single DNPA to select a primary DNPA instance for a cluster of different entities. Each DNPA instance in the illustrated embodiment includes... Figure 4 The components shown communicate with a network manager (not shown). Each DNPA instance for DNPA 655 includes DNPA instances 645a-645n. DNPA instance 645n is selected as the primary DNPA instance for Kubernetes node 635A, and DNPA instance 645a is selected as the primary DNPA instance for Kubernetes node 635B. In some embodiments, the groups of DNPA instances connected to Kubernetes nodes 635A and 635B include different DNPA instances running on different host computer groups. In other embodiments, the number of instances connected to Kubernetes nodes 635A and 635B is the same, but the host computer groups on which they run are different. Additionally, different logical networks 660A and 660B are used to connect worker nodes and DNPA instances to different VPCs or tenants. Figure 4 The logical network 460 is implemented similarly.

[0061] In some embodiments, each DNPA instance implements multiple DNPAs for different tenants or VPCs and receives notifications from the compute deployment agent from worker nodes in different VPCs or belonging to different tenants. In some embodiments, the number of DNPA instances used for these different DNPAs is independent and they run on independent host computer groups (i.e., the number and identity of DNPA instances can be the same or different). Furthermore, different logical networks are used to connect worker nodes and DNPA instances in different VPCs or tenants. Figure 4 The logical network 460 is implemented similarly. In some embodiments, different DNPAs are deployed for different SDN manager clusters, such that for each SDN manager cluster, a set of host computers hosting components associated with that SDN manager cluster implements only one DNPA. Such embodiments reduce the number of components sending requests to a particular SDN manager cluster and avoid overwhelming the SDN manager cluster as described above.

[0062] Figure 7 The diagram conceptually illustrates a process 700 in which DNPA performs a cluster selection of a master cluster manager plugin (CMP) for a group of deployed worker nodes associated with the DNPA and processes cluster-level requests at the selected master CMP. Process 700 begins by receiving (at 705) data about the resources available for each DNPA instance. In some embodiments, this data is about the resources available for each CMP on the DNPA instance. In some embodiments, this data is received based on a request from the master DNPA instance.

[0063] In addition to receiving (at 705) data about the resources available to each DNPA instance, the primary DNPA instance also receives (at 710) data about the clusters of deployed worker nodes. In some embodiments, this data includes the number of deployed clusters. In some embodiments, this data includes data identifying the cluster for which each host computer performs worker nodes (e.g., host 1 performs worker nodes for clusters 1, 4, and 7). In some embodiments, the data about resources and deployed clusters are received in a different order.

[0064] Based on received data regarding the resources available for each DNPA instance and the number of clusters, the DNPA (e.g., the primary DNPA instance) selects (at 715) a primary cluster manager plugin (CMP) to handle events related to each cluster. In some embodiments, different CMPs 484 of different DNPA instances 445 are selected as masters for multiple clusters, a single cluster, and no cluster. The CMP selected as the primary CMP for each cluster then processes (at 720) cluster-level requests for the cluster for which it was selected as the primary CMP. In some embodiments, cluster-level requests include requests to add, remove, or modify the following: (1) the namespace of the cluster, (2) services associated with the cluster (e.g., load balancing and firewalls), (3) the logical network associated with the cluster, and (4) a set of network policies for the cluster. In some embodiments, a primary pod manager plugin (PMP) is not selected because each PMP is responsible for delivering requests related to individual pods running on the same host machine.

[0065] Figure 8 A conceptual illustration of process 800 for parsing and queuing received notifications at a DNPA instance. Process 800 begins by receiving an event notification (at 805) from a compute deployment agent. In some embodiments, this notification is received via the host computer's hyperbus from a compute deployment agent (e.g., in worker node 435) running on the same host computer. In some embodiments, this notification is a CRUD request received from a worker node (i.e., not just the primary worker node) configured to perform cluster components in a VPC. The event notification is received at listener plug-in 485 (at 805) based on prior registration with node 435 (e.g., node 435's API server). In some embodiments, registration is a long-lived connection session with the API server to receive all CRUD events for various CRDs defined for networking.

[0066] As shown in the figure, each group of Kubernetes nodes 435 running on host computers 450a-n communicates via switch 460. In some embodiments, switch 460 is a logical switch implemented via a PFE on the host computer. In some embodiments, switch 460 is a hyperbus connecting machines running on a single host computer but not on other host computers. In some embodiments, logical switch 460 is a single logical switch connecting Kubernetes nodes 435 to each network plugin agent (e.g., a listener plugin for a DNPA instance). In some embodiments using a single logical switch, only the Kubernetes master node (i.e., the compute deployment agent) connects to logical switch 460 to provide notifications of APIs received at the master node. In other embodiments, this notification is received from compute deployment agents (e.g., master worker nodes) running on different host computers. Notifications from compute deployment agents running on other host computers are received via a logical network connecting the compute deployment agents and DNPA instances (e.g., listener plugins for DNPA instances).

[0067] Upon receiving (at 805), the notification is parsed (at 810) to identify a set of individual requests related to cluster-level and pod-level events requiring configuration of network elements. In some embodiments, the notification is an intent-based API that is parsed into a set of events based on a set of custom resource definitions. In some embodiments, parsing the notification includes parsing CRUD requests received from worker nodes executed on the same host machine as the listening plugin for the DNPA instance.

[0068] After parsing (at 810) the notification to identify cluster-level and pod-level requests, process 800 selects (at 812) individual requests from the identified cluster-level and pod-level requests for processing. Process 800 then determines (at 815) whether the selected request is related to the DNPA instance that received the notification. In some embodiments, this parsing and determination is performed by the listening plugin 485. In some embodiments, the parsed request is related to the DNPA instance if (1) it is related to the cluster whose CMP is the master CMP of the cluster, or (2) it is related to a pod on the host computer of the DNPA instance to be created, removed, or modified.

[0069] If process 800 determines that the selected request is not related to the DNPA instance (e.g., if a pod-level request is not related to the host computer, or if a cluster-level request targets a cluster of the DNPA instance's CMP that is not a cluster of its primary CMP), then process 800 continues to determine (at 825) whether there are any additional requests that could potentially be stored in a queue. If the selected request is determined (at 815) to be related to the DNPA instance that received the notification, then the DNPA instance stores (at 820) the request in the DNPA's request queue 486. In some embodiments, the request queue (e.g., request queue 486) is a separate memory structure of the host computer accessed by the DNPA instance's plugins. In some embodiments, the requests determined to be related to the DNPA instance are those that the DNPA instance's CMP and PMP (e.g., CMP 484a and PMP 483a) are responsible for delivering to the network manager (e.g., SDN manager 410).

[0070] After the selected request is stored (at 820), process 800 determines (at 825) whether there are any additional requests that could potentially be stored in the queue. If process 800 determines (at 825) that there are additional requests to process, process 800 returns to operation 812 to select the next request to process. However, if process 800 determines (at 825) that there are no additional requests to process, process 800 terminates. In some embodiments, after parsing the notification (at 810), all identified requests are added to the queue, and the filtering operations described in operations 812-825 are performed at the Pod manager plugin or cluster manager plugin, which retrieves requests from the request queue and filters the retrieved requests to ignore those irrelevant to the DNPA instance.

[0071] Figure 9A conceptual illustration of process 900 for processing queued requests at a DNPA instance is provided. In some embodiments, this process is performed by both a Pod manager plugin (e.g., PMP 483) and a cluster manager plugin (e.g., CMP 484) to handle pod-level and cluster-level requests, respectively. Process 900 begins by selecting (at 905) requests from a request queue (e.g., request queue 486) to be processed and sent to a network manager (e.g., SDN manager 410). After selecting (at 905) the requests to be processed, process 900 retrieves (at 910) metadata associated with the selected request. In some embodiments, retrieving metadata includes retrieving at least a portion of the metadata from the request queue. In some embodiments, additional metadata is retrieved from a worker node or master node of the cluster associated with the request. In some embodiments, the determination of whether the request is associated with the DNPA instance, as described in operation 815 of process 800, is performed instead as part of process 900, and process 900 proceeds to the next operation only if the request is associated with the DNPA instance.

[0072] After retrieving (at 910) the metadata, process 900 generates (at 920) data related to the request to provide to the network manager to implement the requested network (cluster) components. For cluster-level requests, generating this data includes identifying the network elements affected by the cluster-level request at the cluster manager plugin. For example, a request to implement a load balancer or firewall requires generating data to identify the Pods, containers, and / or machines providing load balancing, or generating firewall rules based on firewall policies (e.g., policies specifying groups of endpoints resolved to individual port identifiers by the cluster manager plugin). For Pod-level requests, in some embodiments, the generated data includes at least one port identifier for the requested Pod-level construct. In some embodiments, Pod-level requests do not require the generation of additional data.

[0073] After generating (at 920) any relevant data, process 900 determines (at 925) whether the parent structure or component required to process the request exists. For example, in some embodiments, the parent structure for a namespace is a cluster, and the parent structure for a pod is a namespace. Those skilled in the art will understand that different requests in different embodiments will require different parent structures in a hierarchical network structure, and some non-hierarchical structures will skip this determination. If the process determines (at 925) that the parent structure does not exist, process 900 returns to operation 905 to select the individual request to process. If the request for which the parent structure does not exist is the only unprocessed request, process 900 will attempt to process the request again. In some embodiments, process 900 attempts to process the request periodically or registers for notification of the creation of the required structure or component.

[0074] If process 900 determines (at 925) that the required structure or component exists, the request, retrieved metadata, and any generated data (at 930) are sent to the network manager (e.g., by CMP 484 or PMP 483 using communication agent 482). In some embodiments, the metadata and / or generated data are appended to the request to send it to the network manager. After sending the request and associated data (at 930) to the network manager, process 900 determines (at 935) whether there are any additional requests to process. If there are no additional requests to process, process 900 terminates. In some embodiments, when this process terminates, process 900 is placed in a wait / listen mode and restarts when the next request is added to the request queue by the listener plugin. If there are additional requests to process in the request queue, process 900 returns to operation 905 to select individual requests in the request queue, and the process continues as described above.

[0075] In some embodiments, process 900 is limited to processing and sending requests at a rate not exceeding a rate limit calculated by the primary DNPA instance and distributed to DNPA instances. In some embodiments, these requests are sent to the network manager in the order they are received. In some embodiments, cluster-level requests and pod-level requests are processed separately in the order they are received, and each is limited to a rate limit calculated for that particular type of operation. For example, a DNPA instance with a large number of pending cluster-level requests and a small number of pod-level requests may have a higher rate limit for processing cluster-level requests than it has for processing pod-level requests, and vice versa.

[0076] Figure 10A-10D The diagram illustrates a set of operations performed on two clusters spanning three nodes (e.g., host computers) 1060a-c, where each node executes a DNPA instance, with the DNPA instance DNPA1 executed on node 1 being the primary (active) DNPA instance. Each DNPA instance includes a listener plugin "LA" (columns 1020a-c), a cluster manager plugin "CMP" (columns 1030a-c), and a pod manager plugin "PMP" (columns 1050a-c). Figure 10A-10D Includes columns 1030a-c representing the actions taken by DNPA, represented by DNPA instances 1-3.

[0077] Figure 10AThe illustration depicts a set of actions taken between times T0 and T2 (1011-1013) in response to a request to deploy worker nodes 1 and 2 (WN1 and WN2) for "Cluster-1" on nodes 1 and 3, respectively. In this example, the request is the first request for Cluster-1. At T01011, each listening agent on node 1060, where WNs are to be deployed, receives an event. Based on the received event, the DNPA instance (i.e., DNPA1 1030a), acting as the primary DNPA instance, determines the primary CMP (i.e., CMP1 1040a) for Cluster-1 at time T11012. After selecting the primary CMP for Cluster-1, DNPA1 broadcasts a message to all other DNPA instances at time T21013 to notify them of the primary CMP selected for Cluster-1. In some embodiments, the actual deployment is handled, for example, by the compute manager and controller 117. In some embodiments, CMP1 1040a then transmits the deployment of WNs to the SDN manager for network connectivity.

[0078] Figure 10B The diagram similarly illustrates a set of actions taken at times T3-T5 (1014-1016) in response to a request to deploy worker nodes (WNs) for "Cluster 2" on nodes 2 and 3. In this example, this request is the first request for Cluster 2. At T3 1014, each listening agent on node 1060, where the WN is to be deployed, receives an event. Based on the received event, the DNPA instance (i.e., DNPA1 1030a), which is the primary DNPA instance, determines the primary CMP (i.e., CMP3 1040c) for Cluster 2 at time T4 1015. After selecting the primary CMP for Cluster 2, DNPA1 broadcasts a message to all other DNPA instances at time T5 1016 to notify them of the primary CMP selected for Cluster 2. Figure 10A and 10B As shown, different master CMPs are selected for different clusters. In some embodiments, the actual deployment is handled, for example, by the compute manager and controller 117. In some embodiments, the CMP3 1040c then transmits the WN deployment to the SDN manager for network connection.

[0079] Figure 10CThe diagram illustrates a set of actions taken at times T6 1017 and T7 1018 in response to a request to deploy a namespace (NS) for cluster 1. At T6 1017, each listening agent 1020 of node 1060 executing the components of cluster 1 (i.e., LAI 1020a of node 1 1060a and LA3 1020c of node 3 1060c) receives an event. Based on the received event, LA 1020 determines that the event (i.e., add namespace) is a cluster-level event and determines whether the CMP executed on the same node 1060 is the master of the corresponding cluster. If LA 1020 determines that the CMP executed on the same node 1060 is the master of the corresponding cluster, then LA 1020 adds the request to a queue (e.g., LA1 1020a adds the request to the queue of DNPA instance 1030a). However, if LA 1020 determines that the CMP executing on the same node 1060 is not the master of the corresponding cluster, LA 1020 ignores (rejects) the request and does not add it to the queue (e.g., LA3 1020c rejects the request and does not add it to the queue of DNPA instance 1030c). The CMP acting as the master of the cluster (e.g., CMP11040a) then processes the cluster-level request at T7 1018 to add the namespace.

[0080] Figure 10D The diagram illustrates a set of actions taken at times T8 1019a and T9 1019b in response to requests to deploy Pod 1 and Pod 2 for "Cluster 1" on WN 1 and 2. At T8 1019a, each listening agent 1020 on node 1060 where the Pods are to be deployed receives an event (i.e., LA1 1020a on node 1 1060a receives a request to add "POD1", and LA3 1020c on node 3 1060c receives a request to add "POD2"). Based on the received event, LA 1020 determines that the event (i.e., adding a Pod) is a Pod-level event and adds the request to the queue of DNPA instance 1030 executing on the same node (e.g., the host computer). At T9 1019b, PMP 1050 processes the request to add the Pods to WN.

[0081] Figure 11 and Figure 12 The diagram illustrates two network services that may require configuration data generated at the network manager and / or network controller based on detected events related to machines in the VPC in order to provide services to those machines. Figure 11The illustration shows an example of a distributed load balancer 1100 defined by the NPA for several VIFs associated with several machines 1130 on several host computers 1150. At the request of the NPA, the SDN manager / controller, in some embodiments, configures a hook for the software switch port 1110 to which the VIF 1105 is connected (i.e., the VIF associated with it) to the load balancer 1115 (which executes on the same host computer as the VIF 1105). In some embodiments, a load balancer 1115 is instantiated for each VIF 1105 that requires associated client load balancing operations. In some embodiments, each load balancer 1115 is a service engine provided by a hypervisor executing on the same computer as machine 1130.

[0082] Hooks are configured to direct traffic entering or leaving the VIF-associated machine 1130 (provided by or to the VIF-associated machine 1130) to its corresponding load balancer inlet and / or outlet. Each load balancer 1115 uses a set of load balancing rules (stored in the LB rule storage device 1120) to identify a set of end nodes 1125 that should process the data message flow entering or leaving the machine 1130. In some embodiments, the load balancer 1115 then selects an end node 1125 for each data message flow using load balancing criteria (e.g., a weight value for the round-robin selection of end nodes 1125), and then forwards one or more data messages of the flow to the end node 1125 selected for that flow. As shown, in some embodiments, the load balancing rules and criteria are stored in the LB rule storage device 1120.

[0083] In some embodiments, this selection of the load balancer 1115 for a data message stream can be stateful, such that all data messages of a stream are sent to a single end node 1125. Alternatively, in some embodiments, this selection can be stateless, such that different data messages of the same stream are sent to different end nodes 1125. Additionally, in some embodiments, the load balancer 1115 can be configured to send only the first or first few data messages of the stream to the end node machine 1125.

[0084] In some embodiments, end node 1125 may be a service node for inbound or outbound traffic, or a destination compute node for outbound traffic. End node 1125 may be an engine / machine on the same host computer 1150 as the machine 1130 associated with the client VIF and the load balancer 1115, may be an engine / machine on a different host computer, or may be a standalone device. In some embodiments, end node 1125 is associated with a virtual network address (e.g., a VIP address) or a set of associated network addresses (e.g., a set of associated IP addresses). In some embodiments, end node machine 1125 is a Pod, a VM, and / or a container running on a Pod / VM.

[0085] In some embodiments, when forwarding data messages to end-node machines 1125 residing on the same host computer, the load balancer 1115 forwards these data messages via a software switch 1155 on its host computer 1150. Alternatively, when forwarding data messages to end-node machines 1125 not residing on the same host computer, the load balancer 1115 forwards the data messages via its host computer's software switch 1155 and / or software router (not shown) and an intermediate network structure.

[0086] In some embodiments, the NPA instructs the SDN manager / controller to configure hooks on port 1110 associated with the VIF for use in other middlebox service operations, such as firewall, intrusion detection, intrusion prevention, deep packet inspection, encryption, etc. Figure 12 The illustration shows an example of a port hook configured to direct ingress / egress data message flows from / to the VIF-associated machine 1130 to a firewall engine 1215 on the same host computer 1150 that performs firewall operations on these flows. In some embodiments, these firewall operations determine whether the data message flows should be allowed through a machine or software switch, should be dropped, or should be redirected to another service machine or device.

[0087] As shown in the figure, a plurality of firewall engines 1215 on a plurality of host computers 1150 implement a distributed firewall 1200. In some embodiments, multiple such firewall engines 1215 execute on the same host computer 1150 for distributed firewall operations configured for multiple different VIF-associated machines 1130. To perform their firewall operations, firewall engines 1215 use firewall rules stored in firewall rule storage 1220. In some embodiments, firewall rules are defined by an SDN manager / controller at the request of an NPA. In some embodiments, the NPA defines distributed firewall operations to implement network policy services defined in the Kubernetes API, as well as other firewall operations described in this document.

[0088] As discussed in more detail in U.S. Patent Application No. 16 / 897,652, in some embodiments, the control system 100 may configure service and forwarding rules to enable the VIF-associated machine 1130 to perform server-side or server-side operations. To facilitate the use of the VIF-associated machine 1130 as a service machine or server machine, some embodiments use endpoint grouping rules (CRDs).

[0089] Many of the aforementioned features and applications are implemented as a software process that specifies a set of instructions recorded on a computer-readable storage medium (also known as a computer-readable medium). When these instructions are executed by one or more processing units (e.g., one or more processors, processor cores, or other processing units), they cause the processing unit(s) to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, RAM chips, hard disk drives, EPROMs, etc. Computer-readable media do not include carrier waves and electronic signals transmitted wirelessly or via wired connections.

[0090] In this specification, the term "software" means including firmware residing in read-only memory or an application stored in magnetic storage, which can be read into memory for processor processing. Furthermore, in some embodiments, multiple software inventions may be implemented as sub-parts of a larger program while retaining distinct software inventions. In some embodiments, multiple software inventions may also be implemented as separate programs. Finally, any combination of separate programs that collectively implement the software inventions described herein is within the scope of this invention. In some embodiments, when a software program is installed to run on one or more electronic systems, one or more specific machine implementations define the operations for executing and running the software program.

[0091] Figure 13A computer system 1300 is conceptually illustrated, in which some embodiments of the present invention are implemented. The computer system 1300 can be used to implement any of the host computers, controllers, and managers described above. Therefore, it can be used to perform any of the processes described above. This computer system includes various types of non-transitory machine-readable media and interfaces for various other types of machine-readable media. The computer system 1300 includes a bus 1305, processing units (one or more) 1310, system memory 1325, read-only memory 1330, permanent storage device 1335, input device 1340, and output device 1345.

[0092] Bus 1305 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of computer system 1300. For example, bus 1305 communicatively connects processing unit(s) 1310 to read-only memory 1330, system memory 1325, and permanent storage device 1335.

[0093] From these various memory units, processing units (one or more) 1310 retrieve instructions to be executed and data to be processed to perform the process of the present invention. In different embodiments, the processing units (one or more) may be a single processor or a multi-core processor. Read-only memory (ROM) 1330 stores static data and instructions required by the processing units (one or more) 1310 and other modules of the computer system. On the other hand, permanent storage device 1335 is a read-write memory device. This device is a non-volatile memory unit that stores instructions and data even when the computer system 1300 is turned off. Some embodiments of the present invention use mass storage devices (such as disks or optical disks and their corresponding disk drives) as permanent storage device 1335.

[0094] Other embodiments use a removable storage device (such as a floppy disk, flash drive, etc.) as permanent storage device 1335. Like permanent storage device 1335, system memory 1325 is a read-write memory device. However, unlike storage device 1335, system memory 1325 is volatile read-write memory, such as random access memory. System memory 1325 stores some instructions and data required by the processor during operation. In some embodiments, the processes of the present invention are stored in system memory 1325, permanent storage device 1335, and / or read-only memory 1330. Processing unit (one or more) 1310 retrieves instructions to be executed and data to be processed from these various memory units to perform the processes of some embodiments.

[0095] Bus 1305 is also connected to input and output devices 1340 and 1345. Input device 1340 enables the user to transmit information and make selection requests to the computer system. Input device 1340 includes an alphanumeric keypad and a pointing device (also referred to as a "cursor control device"). Output device 1345 displays images generated by the computer system. Output device 1345 includes printers and display devices such as cathode ray tube (CRT) or liquid crystal display (LCD). Some embodiments include devices that serve as both input and output devices, such as touchscreens.

[0096] Finally, as Figure 13 As shown, bus 1305 also couples computer system 1300 to network 1365 via a network adapter (not shown). In this way, the computer can be part of a computer network (such as a local area network (“LAN”), wide area network (“WAN”), or intranet) or a network of networks (such as the Internet). Any or all components of computer system 1300 can be used in conjunction with this invention.

[0097] Some embodiments include electronic components, such as microprocessors, that store computer program instructions in a machine-readable or computer-readable medium (alternatively referred to as a computer-readable storage medium, machine-readable medium, or machine-readable storage medium). Examples of such computer-readable media include RAM, ROM, read-only optical disc (CD-ROM), recordable optical disc (CD-R), rewritable optical disc (CD-RW), read-only digital versatile optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini SD card, micro SD card, etc.), magnetic and / or solid-state hard disk drives, read-only and recordable... Optical discs, high-density optical discs, any other optical or magnetic media, and floppy disks. Computer-readable media may store computer programs executable by at least one processing unit and include a set of instructions for performing various operations. Examples of computer programs or computer code include machine code generated by a compiler, and files that include higher-level code executed by a computer, electronic component, or microprocessor using an interpreter.

[0098] While the above discussion primarily refers to microprocessors or multi-core processors that execute software, some embodiments are executed by one or more integrated circuits such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuit itself.

[0099] As used in this specification, the terms "computer," "server," "processor," and "memory" refer to electronic or other technical devices. These terms do not include people or groups of people. For the purposes of this specification, the terms "display" or "being displayed" refer to something displayed on an electronic device. As used in this specification, the terms "computer-readable medium," "multiple computer-readable media," and "machine-readable medium" are limited entirely to tangible physical objects that store information in a computer-readable form. These terms do not include any wireless signals, wired download signals, or any other transient or temporary signals.

[0100] While the invention has been described with reference to numerous specific details, those skilled in the art will recognize that it may be embodied in other specific forms without departing from its spirit. Furthermore, although some of the foregoing examples refer to Pods, other embodiments use containers other than Pods. Therefore, those skilled in the art will understand that the invention is not limited to the foregoing illustrative details but is to be defined by the appended claims.

Claims

1. A method for connecting deployed machines to a Virtual Private Cloud (VPC) in an Availability Zone (AZ): Multiple network plug-in agents are deployed on multiple host computers, which execute multiple machines deployed by multiple compute deployment agents. The multiple network plug-in agents are different instances of Distributed Network Plug-in Agent (DNPA), and each network plug-in agent includes a request queue, a cluster manager plug-in, a pod manager plug-in, and a communication agent. Configure each specific deployed network plug-in agent to receive event notifications from a set of compute deployment agents running on the host computer of that specific deployed network plug-in agent; Receive notifications from the deployed network plug-in agent regarding events related to the deployed machine, and In response to the received notification, configure network elements to connect to one or more groups of deployed machines. The configuration of each specific deployed network plug-in agent includes: (i) configuring a request queue for each specific network plug-in agent to store requests related to notifications received by the network plug-in agent; (ii) configuring a cluster manager plug-in for each specific network plug-in agent to process requests related to cluster-level requests stored in the request queue, wherein cluster-level requests are requests related to worker node clusters in a plurality of worker node clusters running on the plurality of host computers; (iii) configuring a pod manager plug-in for each specific network plug-in agent to process requests related to pod-level requests stored in the request queue, wherein the pod-level requests are related to worker nodes running on the host computers; and (iv) configuring a communication proxy for each specific network plug-in agent to transmit the requests processed by the cluster manager plug-in and the pod manager plug-in to a network manager, which configures the network elements in the availability zone based on the requests processed by the cluster manager plug-in and the pod manager plug-in.

2. The method as described in claim 1, wherein, Configuring network elements to connect to one or more groups of deployed machines includes: instructing a network controller to configure the network elements to connect to the one or more groups of deployed machines.

3. The method as described in claim 1, wherein, Configuring each specific deployed network plug-in agent to receive event notifications from a set of compute deployment agents includes registering that specific network plug-in agent with a set of application programming interface (API) processors of the set of compute deployment agents that are executed on the host computer of the particular deployed network plug-in agent and which receives the request from the deployment machine.

4. The method of claim 3, wherein, The compute deployment agent is the master worker node of the Kubernetes worker node cluster, and the master worker node receives requests to modify a set of machines in the Kubernetes worker node cluster.

5. The method of claim 4, wherein, The request to modify the set of machines in the Kubernetes cluster includes at least one of the following: (1) a request to add a machine to the set of machines, (2) a request to remove a machine from the set of machines, and (3) a request to modify a deployed machine, the set of machines including at least one of containers and pods that need to connect to the VPC.

6. The method of claim 5, wherein, Configuring network elements to connect to one or more groups of deployed machines includes: (1) assigning a set of network addresses to machines in the group of deployed machines, and (2) updating a set of forwarding elements of the VPC to connect to the group of deployed machines using at least one of the set of network addresses.

7. The method of claim 4, wherein, A specific master worker node in a specific Kubernetes cluster monitors a set of other worker nodes in the specific Kubernetes cluster, and the API processor receives requests based on at least one of the monitoring of the set of other worker nodes and instructions received from outside the specific Kubernetes cluster.

8. The method of claim 7, wherein, The request received at the API processor is based on the detection at the master worker node of the Kubernetes cluster that a machine in the Kubernetes cluster has failed and the determination that a replacement machine should be deployed, wherein the request received at the API processor is a request for deploying the replacement machine.

9. The method of claim 1, wherein, One of the multiple DNPA instances running on the multiple host computers is designated as the primary DNPA instance for the DNPA, and The primary DNPA instance (1) receives data about the number of pending requests stored in a request queue associated with a DNPA instance executing on each of the plurality of host computers, (2) determines the maximum rate at which requests are processed at the network manager, (3) determines the total number of pending requests related to the deployment of the machine, (4) calculates a rate limit for each of the plurality of DNPA instances, the rate limit specifying the rate at which the DNPA instance is allowed to transmit requests related to the deployment of the machine to the network manager, and (5) distributes the rate limit calculated for each particular DNPA instance to that particular DNPA instance.

10. The method of claim 9, wherein, The primary DNPA instance calculates the rate limit for each specific DNPA instance based on the following: (1) the number of pending requests associated with the specific DNPA instance, (2) the maximum rate at which requests are processed at the manager, and (3) the total number of pending requests related to the deployment of the machine.

11. The method of claim 9, wherein, The primary DNPA instance selects a cluster manager plugin of a specific DNPA instance as the primary cluster manager plugin for at least one of the plurality of worker node clusters. The primary cluster manager plugin for each specific worker node cluster is the only cluster manager plugin that handles cluster-level requests related to that specific worker node cluster.

12. The method of claim 11, wherein, The primary DNPA instance receives data about the resources available on each of the plurality of host computers, and the primary cluster manager plugin for each worker node is selected based on the received data about the resources available on each of the plurality of host computers.

13. The method of claim 11, wherein: The DNPA is associated with a set of Kubernetes clusters that include worker nodes; Each Kubernetes cluster has a master worker node that receives API requests related to that Kubernetes cluster; The master worker node of each Kubernetes cluster in the set of Kubernetes clusters is connected to a logical network, which in turn connects the master worker node to a network plug-in agent deployed on the multiple host computers. and Each network plug-in agent executing on each of the plurality of host computers receives a notification from each master worker node and determines whether to add the relevant request to the request queue on the same host computer as the network plug-in agent based on whether the notification is related to any of the following: (1) a cluster-level request related to a cluster on which the cluster manager on the same host computer has been designated as the master cluster manager of the cluster, or (2) a pod-level request related to a worker node executing on the same host computer.

14. The method of claim 1, wherein, The notification includes a forwarded intent-based API request received at an application programming interface (API) processor for modifying a set of machines, and the network plug-in agent parses the forwarded API request to generate at least one of a first set of cluster-level requests and a second set of pod-level requests, thereby identifying at least one of the cluster-level requests and pod-level requests to be stored in a request queue on the host computer.

15. The method of claim 1, wherein, The notification includes a forwarded CRUD (Create, Read, Update, and Delete) request received at a worker node executing on the host computer for modifying a group of machines, and the network plug-in agent parses the forwarded CRUD request to identify at least one of cluster-level requests and pod-level requests to be stored in the request queue on the host computer.

16. The method of claim 9, wherein, Each DNPA instance processes requests stored in the request queue of the host computer on which the DNPA instance executes, at a rate not greater than the rate limit received from the primary DNPA instance, the processing being performed in the order in which these requests were stored in the request queue on the host computer.

17. The method of claim 9, wherein, The data regarding the number of pending requests includes data regarding the number of pending cluster-level requests and the number of pending pod-level requests. The rate limit is a first rate limit specifying the rate at which a DNPA instance is allowed to transmit cluster-level requests related to the deployment of the machine to the network manager, and the primary DNPA instance calculates a second rate limit for each of the plurality of DNPA instances, the second rate limit specifying the rate at which the DNPA instance is allowed to transmit pod-level requests related to the deployment of the machine to the network manager.

18. The method of claim 9, wherein, The DNPA is associated with a set of Kubernetes clusters including worker nodes, and each worker node on the host computer is connected to a hyperbus on the host computer that connects the worker node to the network plug-in agent, wherein notifications sent from each worker node to the network plug-in agent are sent using the hyperbus.

19. A machine-readable medium storing a program that, when implemented by at least one processing unit, implements the method as described in any one of claims 1-18.

20. An electronic device, comprising A set of processing units; and A machine-readable medium storing a program that, when executed by at least one of the processing units, implements the method as described in any one of claims 1-18.

21. A system comprising components for implementing the method as described in any one of claims 1-18.

22. A computer program product comprising instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-18.

Images