Scaling IP addresses in overlay networks

Abstract

This provides a method for scaling IP addresses in an overlay network without using a load balancer. [Solution] An overlay IP address is attached to multiple compute instances via a virtual network interface card (VNIC) associated with multiple compute instances. Traffic directed to the multi-attached IP address is distributed across the multiple compute instances. The overlay IP address is scaled using ECMP technology in the overlay network. In the forwarding table, the scaled IP address is associated with multiple next-hop paths to multiple network virtualization devices (NVDs) associated with multiple compute instances. One of the multiple next-hop paths is selected to route a specific packet directed to the overlay IP address.

Description

Technical Field

[0001] Reference to Related Applications This international patent application claims priority to U.S. Patent Application No. 17 / 175,285, filed on February 12, 2021, titled "Scaling IP Addresses in Overlay Networks", the entire disclosure of which is incorporated herein by reference for all purposes.

Background Art

[0002] Background In a physical networking environment, horizontal scaling of network addresses such as Internet Protocol (IP) addresses is typically performed using a load balancer (e.g., a layer 4 to layer 7 load balancer). The load balancer advertises an address (sometimes referred to as a virtual IP address or VIP) that a client can communicate with. For example, a client can send a packet to this VIP (i.e., the destination IP address of the packet is the VIP address of the load balancer). Then, the load balancer selects a backend server or node to receive traffic from among a plurality of available backend nodes each having its own IP address. In this way, the load balancer distributes or spreads the traffic across the plurality of backend nodes. Responses generated by the backend nodes are first received by the load balancer and then communicated to the client. There are several problems with the use of load balancers. Load balancers are very costly and complex to configure, and as a result, it becomes difficult to scale. Furthermore, since load balancers must actively participate in the communication between the client and the backend nodes, a significant latency is added to the communication when using those load balancers. ​​[Overview of the Initiative]

[0003] overview This disclosure relates, in general, to a virtualized cloud networking environment, and more specifically, to a technique for scaling IP addresses in an overlay network. IP scaling is achieved without the use of load balancers. This specification describes various embodiments, including methods, systems, and non-temporary computer-readable storage media for storing programs, code, or instructions executable by one or more processors.

[0004] In certain embodiments, the cloud infrastructure (referred to as cloud service provider infrastructure or CSPI) The technology implemented in Kucha provides IP address scaling in the overlay network by enabling multiple compute instances in the cloud to be associated with the same single overlay IP address. This makes it possible to horizontally scale the overlay IP address across multiple compute instances without using a load balancer.

[0005] In certain implementations, an overlay IP address can be attached to multiple compute instances. This is done by attaching the overlay IP address to multiple virtual network interface cards (VNICs) and associating the VNICs with multiple compute instances. Duplicate IP addresses attached to a computing instance are called multi-attach IP addresses. Traffic directed to these multi-attach IP addresses is then distributed across multiple compute instances without the need for a load balancer. Processing can be performed in stateful or stateless mode.

[0006] In some other implementations, Equal-Cost Multi-Path (ECMP) technology in overlay networks scales overlay IP addresses. It is used for this purpose. In this implementation, the forwarding is performed by a network virtualization device (NVD) to forward packets. In the table, a scaling overlay IP address is associated with multiple next-hop paths to multiple NVDs associated with multiple compute instances. When a particular packet destined for an overlay IP address is received by an NVD and should be routed, the NVD uses the forwarding table to select one of the multiple next-hop paths for routing the packet. Here, the selected path leads from the multiple compute instances associated with the overlay IP address to a specific NVD associated with that particular compute instance. This makes it possible to distribute packets destined for an overlay IP address across multiple compute instances.

[0007] In some embodiments, a network virtualization device (NVD) may receive a first packet whose destination address is a first Internet Protocol (IP) address. The NVD may determine that multiple IP addresses are associated with the first IP address. The first IP address is associated with multiple compute instances, and the multiple IP addresses include the IP addresses of multiple NVDs associated with the multiple compute instances. The first NVD may select a specific IP address from the multiple IP addresses. The specific IP address belongs to a specific NVD from the multiple NVDs, and the specific NVD is associated with a first compute instance from the multiple compute instances. The first NVD may then communicate the first packet to the specific IP address. The first packet may then be communicated from the specific NVD to the first compute instance. The first IP address may be an overlay IP address, and the addresses within the multiple IP addresses may be physical IP addresses.

[0008] In certain embodiments, the first NVD may determine that multiple IP addresses are associated with a first IP address based on IP mapping information stored by the first NVD. The IP mapping information indicates that the first IP address maps to multiple IP addresses. In some other embodiments, the first NVD may determine that multiple IP addresses are associated with a first IP address based on information in a forwarding table. The forwarding table includes information indicating that multiple next-hop routes are associated with the first IP address, and the multiple next-hop routes identify multiple IP addresses.

[0009] The first NVD may select a specific IP address from multiple IP addresses using various techniques. In certain implementations, the first NVD may generate a hash value based on the content of one or more fields of the first packet. The generated hash value can then be used to select a specific IP address from multiple IP addresses. In certain embodiments, consistent hashing techniques may be used to generate the hash value.

[0010] In a particular implementation, the first NVD may first determine that the first packet belongs to the first network flow. The first NVD may then identify the first entry corresponding to the first network flow from the flow information stored by the first NVD. The first NVD may then determine that a particular IP address is designated in the first entry as being associated with the first network flow. In some other cases, the first NVD may determine that the flow information stored by the first NVD does not contain an entry corresponding to the first network flow. In such cases, the first NVD may use the first selection technique to select a particular IP address from a group of IP addresses. The first NVD may add an entry to the flow information that identifies the first network flow and associates the particular IP address with the first network flow.

[0011] The first NVD may receive packets from multiple clients directed to the same first IP address. For example, the first NVD may receive a second packet from another client, and the second packet is also directed to the first IP address. Unlike the first packet, the first NVD may select a second specific IP address from multiple IP addresses. The second specific IP address belongs to a second specific NVD from multiple NVDs, and the second specific NVD is associated with a second compute instance from multiple compute instances. The first NVD may then communicate the second packet to the second specific IP address. The first packet may be received from a first client, and the second packet may be received from a second client. The first client is different from the second client.

[0012] In certain embodiments, packets may also originate from compute instances within multiple compute instances associated with the same first IP address. For example, a particular NVD from multiple NVDs may receive a new packet from the first compute instance, the new packet identifying the destination IP address for a destination client. The particular NVD can then determine whether the new packet is for a new connection or communication to be established between the first compute instance and the destination client, or whether the new packet is a response packet sent by the first compute instance in response to a packet previously received by the first compute instance from the destination client. If it is determined that the new packet is for a new connection or communication to be established between the first compute instance and the destination client, an ephemeral port value may be identified so as to be used for the new packet. The source port field of the new packet may be set to the identified ephemeral port value. The new packet may then be communicated from the particular NVD to the NVD associated with the destination client.

[0013] Various techniques can be used to select ephemeral port values. In one embodiment, the ephemeral port value to be used for a new packet is selected from a range of ephemeral port values ​​assigned to a first compute instance. Ephemeral port values ​​within the range assigned to the first compute instance are different from ephemeral port values ​​assigned to other compute instances in a group of compute instances. In another embodiment, a particular NVD may use a service to identify ephemeral port values. For example, a particular NVD may send a request to a service requesting one or more ephemeral port values. In response, the particular NVD may receive a set of one or more ephemeral port values ​​from the service. The source port field of the new packet may then be set to a value from that set of one or more ephemeral port values. In some cases, a packet received from the first compute instance may be determined to be a response packet. The response packet may be communicated from the particular NVD to the NVD associated with the destination client.

[0014] Various different techniques can be used to determine whether a new packet is for a new connection or a response packet. For example, a new packet containing an acknowledgment In some cases, this new packet may be identified as a response packet. As another example, if the value in the source port field of the new packet is not an ephemeral port value, the new packet may be identified as a response packet. As yet another example, if a particular NVD determines that the new packet is part of an existing network flow, this new packet may be identified as a response packet.

[0015] This summary is not intended to identify any significant or essential features of the claimed subject matter, nor is it intended to be used alone to determine the scope of the claimed subject matter. The subject matter should be understood by referring to the appropriate portions of the entire specification of this disclosure, any or all of the drawings, and each claim.

[0016] The above description, along with other features and embodiments, will become clearer with reference to the following specification, the appended claims, and the appended drawings.

[0017] Brief explanation of the drawing Please refer to the attached diagram for a detailed explanation of specific examples below. [Brief explanation of the drawing]

[0018] [Figure 1] This is a high-level diagram of a distributed environment showing a virtual or overlay cloud network hosted by a cloud service provider infrastructure, according to a specific embodiment. [Figure 2] This is an architectural schematic diagram showing the physical components of the physical network within the CSPI according to a specific embodiment. [Figure 3] This figure shows an exemplary configuration of a CSPI in which a host machine is connected to multiple network virtualization devices (NVDs) according to a specific embodiment. [Figure 4] This figure shows the connection between a host machine and an NVD that provides I / O virtualization to support multi-tenancy, according to a specific embodiment. [Figure 5] This is a schematic block diagram showing the physical network provided by CSPI according to a specific embodiment. [Figure 6A] This figure shows an example of a distributed virtualization multi-tenant cloud environment that can be hosted by infrastructure provided by a cloud service provider (CSP) (Cloud Service Provider Infrastructure (CSPI)) according to a specific embodiment, and that enables scaling of overlay IP addresses using multi-attach IP addresses in stateful mode processing without the use of a load balancer. [Figure 6B]A diagram showing an example of a distributed virtualized multi-tenant cloud environment that can be hosted by a CSPI provided by a cloud service provider (CSP), and enables scaling of overlay IP addresses using multi-homed IP addresses in a stateless mode of processing without using a load balancer. [Figure 7A] A simplified flowchart showing the process executed to scale an overlay IP address using a multi-homed IP address using a stateful mode of processing according to a particular embodiment. [Figure 7B] A simplified flowchart showing the process executed to scale an overlay IP address using a multi-homed IP address using a stateful mode of processing according to a particular embodiment. [Figure 8] A simplified flowchart showing the process for scaling an overlay IP address using a multi-homed IP address using a stateless mode of processing according to a particular embodiment. [Figure 9A] A simplified flowchart showing the process for enabling communication from a server computing instance attached to a multi-homed IP address to one or more clients according to a particular embodiment. [Figure 9B] A simplified flowchart showing the process for enabling communication from a server computing instance attached to a multi-homed IP address to one or more clients according to a particular embodiment. [Figure 10] A diagram showing an example of a distributed virtualized multi-tenant cloud environment that can be hosted by an infrastructure provided by a cloud service provider (CSP) (cloud service provider infrastructure (CSPI)), and enables scaling of IP addresses using the ECMP technique without using a load balancer. [Figure 11]This is a simplified flowchart illustrating the process of scaling IP addresses within an overlay network using ECMP technology, according to a specific embodiment. [Figure 12] This block diagram shows one pattern for implementing cloud infrastructure as a service system, according to at least one embodiment. [Figure 13] This block diagram shows another pattern for implementing cloud infrastructure as a service system, following at least one embodiment. [Figure 14] This block diagram shows another pattern for implementing cloud infrastructure as a service system, following at least one embodiment. [Figure 15] This block diagram shows another pattern for implementing cloud infrastructure as a service system, following at least one embodiment. [Figure 16] This is a block diagram illustrating an exemplary computer system according to at least one embodiment. [Modes for carrying out the invention]

[0019] Detailed explanation The following description includes specific details to enable a full understanding of certain embodiments for illustrative purposes. However, it will be apparent that various embodiments can be implemented without these specific details. Figures and descriptions are not intended to be limiting. The term “exemplary” is used herein to mean “serving as an example, case, or illustration.” No embodiment or design described herein as “exemplary” should be construed as necessarily preferable or advantageous to other embodiments or designs.

[0020] This disclosure describes a technique for scaling IP addresses in a virtualized cloud networking environment. In certain embodiments, overlay IP addresses are scaled in a virtualized networking environment without the use of load balancers, thereby avoiding various problems associated with the use of load balancers.

[0021] As shown in the background technology section, in a physical networking environment, horizontal scaling of Internet Protocol (IP) addresses is typically performed using load balancers (e.g., Layer 4 to Layer 7 load balancers). The load balancer advertises the VIP addresses that clients can communicate with. For example, a client can send packets directed to a VIP, i.e., the destination IP address of the packet is the load balancer's VIP address. The load balancer then selects a specific backend server or node from among several available backend nodes, each having its own IP address, to receive the traffic. In this way, the load balancer distributes or spreads the traffic directed to the VIP across multiple backend nodes. Responses generated by the backend nodes are also first received by the load balancer before being communicated to the client. A load balancer must actively participate in communication between clients and backend nodes. For example, a packet received by a load balancer will have the load balancer's destination IP address, i.e., its VIP. The load balancer must change this to the IP address of the selected backend node to which the packet should be sent before the packet is sent to that backend node. Similarly, when a response packet enters the load balancer from a backend node, that packet will have the source IP address of that backend node. Upon receiving the response packet, the load balancer must change the source IP address to that of its VIP and then communicate the packet to the client. A load balancer also needs to perform flow tracking, which involves tracking the network flow that the load balancer is processing. This active participation of the load balancer in communication can add undesirable latency to the communication. Load balancers are very expensive and complex to configure, and as a result, they can be difficult to scale and may not function in complex ways.

[0022] In certain embodiments, a technology implemented in a cloud infrastructure (referred to as Cloud Service Provider Infrastructure or CSPI) provides IP address scaling in an overlay network by enabling multiple compute instances in the cloud to be associated with the same single overlay IP address. This makes it possible to horizontally scale an overlay IP address across multiple compute instances without the use of load balancers. Various different technologies are disclosed that enable associating a single overlay IP address with multiple compute instances and enabling packets to communicate appropriately between these multiple compute instances.

[0023] In certain implementations, an overlay IP address can be attached to multiple compute instances. This is done by attaching the overlay IP address to multiple virtual network interface cards (VNICs) and associating the VNICs with multiple compute instances. The duplicate IP address attached to multiple compute instances is called a multi-attach IP address. Traffic directed to the multi-attach IP address is then distributed across the multiple compute instances without the need for a load balancer.

[0024] As a use case, a CSPI customer may provide cloud services using resources provided by CSPI. These cloud services may be accessible by clients or consumers of the service using a specific overlay IP address. Using the teachings of this disclosure, multiple compute instances (also referred to as server compute instances, as they provide the service) allocated by the customer to provide the cloud services may be attached to a specific overlay IP address. When a client of the service sends a request to access the service to a specific overlay IP address, the network virtualization device (NVD) within CSPI is configured to perform a process to select a specific server compute instance from among the multiple server compute instances attached to the multi-attach IP address to engage with the request, and the client request is then communicated to the selected server compute instance. Different server compute instances may be selected for packets received from different client requests, all destined for the same multi-attach IP address. In this way, traffic from clients destined for that specific overlay IP address is spread across multiple server compute instances, all without the use of load balancers.

[0025] In an embodiment that implements a multi-attach IP address, the multi-attach overlay I The process of forwarding a packet addressed to a P address to a compute instance attached to a multi-attach IP address can be performed in stateful mode or stateless mode. In stateful mode, the NVD, which facilitates the communication of packets destined for the multi-attach IP address, stores network flow information and uses it to identify where the packet should be forwarded, thereby facilitating the transmission of packets to a specific server compute instance attached to the multi-attach IP address. In stateless mode, the NVD does not store or use network flow information. Instead, in stateless mode, one or more headers of the packet destined for the multi-attach IP address are hashed, and the resulting hash value is used to select a specific compute instance from among several compute instances to which the packet should be sent. The packet is then forwarded to the selected compute instance. In both stateless and stateful modes, the same network flow (e.g., Transmission Control Protocol (TCP) network flow, user) is used. Packets belonging to the Datagram Protocol (UDP) network flow are forwarded to the same compute instance from among multiple compute instances with multi-attached IP addresses for processing consistency.

[0026] In some other implementations, Equal-Cost Multipath (ECMP) technology in the overlay network is used to scale overlay IP addresses. In such implementations, the scaling overlay IP address is associated with multiple next-hop paths to multiple NVDs associated with multiple compute instances in the forwarding table used by the NVD to forward packets. When a particular packet destined for an overlay IP address is received by the NVD and should be routed, the NVD uses the forwarding table to select one of the multiple next-hop paths for routing the packet. The selected path leads from the multiple compute instances associated with the overlay IP address to a specific NVD associated with that particular compute instance. This makes it possible to distribute packets destined for an overlay IP address across multiple compute instances.

[0027] In certain embodiments, for both multi-attach IP address implementations and ECMP in overlay network implementations, the infrastructure enables compute instances (referred to as client compute instances) to send packets to overlay IP addresses associated with multiple compute instances, either as multi-attach IP addresses or by ECMP technology, and the packets are appropriately forwarded to a specific compute instance from among the multiple compute instances. Thus, a client compute instance can initiate communication using a single overlay IP address to compute the instance associated with that single overlay IP address. The compute instance receiving the packet can then respond to the packet, for example, by sending a response packet to the client compute instance. In addition, a compute instance associated with a single overlay IP address can also initiate new communication or connections to a client (i.e., not responses). This is not possible with conventional load balancers, where connections to compute instances located behind the load balancer must be initiated externally and cannot be initiated by instances located behind the load balancer. A client compute instance that is the target of such a new connection or communication can respond to that communication, and the client response is forwarded to the appropriate compute instance that sent the initial communication.

[0028] Examples of virtual networking architectures The term "cloud service" generally refers to a service provided by a Cloud Service Provider (CSP). Cloud services refer to services that users or customers can access on demand (for example, via a subscription model) using the systems and infrastructure (cloud infrastructure) provided by the CSP. Typically, the servers and systems that make up the CSP's infrastructure are separate from the customer's own on-premises servers and systems. Therefore, customers can use cloud services provided by the CSP without having to separately purchase hardware and software resources for the service. Cloud services are designed to provide subscribers with easy and scalable access to applications and computing resources without requiring them to invest in procuring the infrastructure used to provide the service.

[0029] There are several cloud service providers that offer various types of cloud services. Cloud services include SaaS (Software-as-a-Service), PaaS (Platform-as-a-Service), IaaS (Infrastructure-as-a-Service), and others. Includes various different types or models.

[0030] A customer can subscribe to one or more cloud services provided by a CSP. A customer may be any entity, such as an individual, organization, or company. When a customer subscribes to or registers for a service provided by a CSP, a tenant or account is created for that customer. The customer can then access one or more subscribed cloud resources associated with this account.

[0031] As mentioned above, IaaS (Infrastructure as a Service) is a single specific type of This is a cloud computing service. In the IaaS model, the CSP provides infrastructure (called Cloud Service Provider Infrastructure or CSPI) that customers can use to build their own customizable networks and deploy their customer resources. Therefore, the customer's resources and network are hosted in a distributed environment by the infrastructure provided by the CSP. This differs from traditional computing where the infrastructure provided by the customer hosts the customer's resources and network.

[0032] A CSPI may include interconnected high-performance computing resources, including various host machines, memory resources, and network resources, forming a physical network also known as an underlay network. The resources of the CSPI may be distributed across one or more data centers that can be geographically dispersed across one or more geographical regions. Virtualization software can run on these physical resources to provide a virtualized distributed environment. Virtualization creates an overlay network (also known as a software-based network, software-defined network, or virtual network) on top of the physical network. The CSPI physical network provides the foundation for creating one or more overlay or virtual networks on top of the physical network. The physical network (or underlay network) includes physical network devices such as physical switches, routers, computers, and host machines. An overlay network is a logical (or virtual) network that operates on top of the physical underlay network. A given physical network can support one or more overlay networks. Overlay networks typically use encapsulation techniques to distinguish traffic belonging to different overlay networks from each other. A virtual network or overlay network is also called a virtual cloud network (VCN). A virtual network is a software virtual network. Virtualization technologies (e.g., hypervisors, network virtualization devices (NVDs) (e.g., smart NICs)) implement virtualization functions, top-of-rack (T) A network abstraction layer can run on top of a physical network, implemented using an OR (Switch), a smart TOR (which implements one or more functions performed by an NVD), and other mechanisms. Virtual networks can take many forms, including peer-to-peer networks and IP networks. A virtual network is typically either a Layer 3 IP network or a Layer 2 VLAN. Such a virtual network or overlay network is often called a virtual Layer 3 network or an overlay Layer 3 network. Examples of protocols developed for virtual networks include IP-in-IP (or GRE (Generic Routing Encapsulation)), Virtual Extensible LAN (VXLAN - IETF RFC7348), and virtual plastic This includes virtual networks (VPNs) (e.g., MPLS Layer 3 Virtual Private Network (RFC4364)), VMware's NSX, and Geneve (Generic Network Virtualization Encapsulation).

[0033] In the case of IaaS, the infrastructure provided by the CSP (CSPI) can be configured to deliver virtualized computing resources over a public network (e.g., the internet). In the IaaS model, the cloud computing service provider can host infrastructure components (e.g., servers, storage, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer)). In some cases, the IaaS provider may also provide various services associated with these infrastructure components (e.g., billing, monitoring, logging, security, load balancing, and clustering). Because these services are policy-driven, IaaS users can maintain application availability and performance by implementing policies to facilitate load balancing. The CSPI provides infrastructure and a set of complementary cloud services, enabling customers to build and run a wide range of applications and services in a highly available hosted distributed environment. The CSPI provides high-performance computing resources and capabilities, as well as storage capacity, on a flexible virtual network that can be securely accessed from various network locations, such as the customer's on-premises network. When a customer subscribes to or registers for an IaaS service provided by a CSP, the tenancy created for that customer becomes a securely isolated partition within the CSP, allowing customers to create, organize, and manage their own cloud resources.

[0034] Customers can build their own virtual networks using the computing, memory, and networking resources provided by CSPI. On these virtual networks, one or more customer resources or workspaces, such as compute instances, can be placed. Loads can be deployed. For example, a customer can use resources provided by CSPI to build one or more customizable private virtual networks called Virtual Cloud Networks (VCNs). On a customer VCN, a customer can deploy one or more customer resources, such as compute instances. Compute instances can take the form of virtual machines, bare metal instances, etc. Thus, CSPI provides the infrastructure and a set of complementary cloud services that enable customers to build and run a wide range of applications and services in a highly available virtual host environment. Customers do not manage or control the underlying physical resources provided by CSPI, but they do control the operating system, storage and deployed applications, and, in some cases, have limited control over select networking components (e.g., firewalls).

[0035] CSP provides a console that enables customers and network administrators to configure, access, and manage resources deployed in the cloud using CSPI resources. This is possible. In certain embodiments, the console provides a web-based user interface that can be used to access and manage the CSPI. In some embodiments, the console is a web-based application provided by the CSP.

[0036] CSPI can support single-tenancy or multi-tenancy architectures. In a single-tenancy architecture, software (e.g., applications, databases) or hardware components (e.g., host machines or servers) serve a single customer or tenant. In a multi-tenancy architecture, software or hardware components serve multiple customers or tenants. Therefore, in a multi-tenancy architecture, CSPI resources are shared among multiple customers or tenants. In multi-tenancy situations, CSPI incorporates safeguards and precautions to ensure that each tenant's data is isolated and not visible to other tenants.

[0037] In a physical network, a network endpoint (endpoint) refers to a computing device or system that is connected to the physical network and communicates bidirectionally with the connected network. A network endpoint in a physical network may be connected to a local area network (LAN), a wide area network (WAN), or other types of physical networks. Examples of traditional endpoints in a physical network include modems, hubs, bridges, switches, routers, and other networking devices, as well as physical computers (or host machines). Each physical device in a physical network has a fixed network address that can be used to communicate with that device. This fixed network address may be a Layer 2 address (e.g., a MAC address), a fixed Layer 3 address (e.g., an IP address), etc. In a virtualized environment or virtual network, endpoints may include various virtual endpoints, such as virtual machines hosted by components of the physical network (e.g., hosted by a physical host machine). These endpoints in a virtual network are addressed by overlay addresses, such as overlay Layer 2 addresses (e.g., overlay MAC addresses) and overlay Layer 3 addresses (e.g., overlay IP addresses). Network overlays provide flexibility by allowing network administrators to move overlay addresses associated with network endpoints using software management (e.g., through software implementing the control plane of the virtual network). Therefore, unlike physical networks, in virtual networks, network management software can be used to move overlay addresses (e.g., overlay IP addresses) from one endpoint to another. Because virtual networks are built on top of physical networks, both the virtual network and the underlying physical network are involved in communication between the components of the virtual network.To facilitate such communication, each component of the CSPI is configured to learn and store mappings that map the overlay addresses of the virtual network to the actual physical addresses of the underlying network, or vice versa. These mappings facilitate communication. To facilitate routing within the virtual network, customer traffic is encapsulated.

[0038] Therefore, physical addresses (e.g., physical IP addresses) are associated with components of a physical network, while overlay addresses (e.g., overlay IP addresses) are associated with entities in a virtual network or overlay network. Physical IP addresses are associated with physical devices within the underlying network or physical network. A physical IP address is an IP address associated with an entity (e.g., a network device). For example, each NVD has a physical IP address associated with it. An overlay IP address is an overlay address associated with an entity in the overlay network, such as a compute instance in a customer's virtual cloud network (VCN). Two different customers or tenants, each with their own private VCN, may, in some cases, use the same overlay IP address within their VCNs without knowing anything about each other. Both physical IP addresses and overlay IP addresses are real IP addresses. These are different from virtual IP addresses, which are typically a single IP address that represents or maps to multiple real IP addresses. A virtual IP address provides a one-to-many mapping between a virtual IP address and multiple real IP addresses. For example, a load balancer may use a VIP to map to or represent multiple servers, each having its own real IP address.

[0039] The cloud infrastructure or CSPI is physically hosted in one or more data centers in one or more regions worldwide. The CSPI may include physical network or underlayment network components and virtualization components of a virtual network built on the physical network components (e.g., virtual networks, compute instances, virtual machines). In certain embodiments, the CSPI is a realm, CSPI resources are organized and hosted within regions and available domains. A region is typically a local geographical area containing one or more data centers. Regions are generally independent of each other and may be separated by vast distances, for example, across countries or continents. For example, one region may be in Australia, another in Japan, and yet another in India. CSPI resources are divided among these regions such that each region has an independent subset of CSPI resources. Each region may provide a set of core infrastructure services and resources, such as computing resources (e.g., bare metal servers, virtual machines, containers and related infrastructure), storage resources (e.g., block volume storage, file storage, object storage, archive storage), networking resources (e.g., virtual cloud networks (VCNs), load balancing resources, connectivity to on-premises networks), database resources, edge networking resources (e.g., DNS), access management and monitoring resources, etc. Each region generally has multiple paths to connect to other regions within the realm.

[0040] Generally, applications are deployed in the region where they are most frequently used (i.e., on infrastructure relevant to that region) because using nearby resources is faster than using distant resources. Applications may also be deployed in different regions for various reasons, such as redundancy to mitigate the risks of large-scale weather systems or region-wide events like earthquakes, or redundancy to meet various requirements for legal jurisdictions, tax domains, and other business or social standards.

[0041] Data centers within a region may be further organized and subdivided into availability domains (ADs). An availability domain is one or more domains located within a given region. This can correspond to the data centers mentioned above. A region may consist of one or more available domains. In such a distributed environment, CSPI resources are region-specific, such as virtual cloud networks (VCNs), or domain-specific, such as compute instances.

[0042] ADs within a single region are configured to be fault-tolerant, isolated from one another, and configured to be extremely unlikely to fail simultaneously. High availability is achieved by using ADs that do not share critical infrastructure resources such as networking, physical cabling, cabling routes, and cabling entry points, so that the failure of one AD within a region has little impact on the availability of other ADs within the same region. By connecting ADs within the same region with a low-latency, high-bandwidth network, high-availability connectivity to other networks (e.g., the internet, customer on-premises networks) can be provided, and a replication system for both high availability and disaster recovery can be built across multiple ADs. Cloud services utilize multiple ADs to ensure high availability and protect against resource failures. As the infrastructure provided by the IaaS provider grows, more regions and ADs may be added while adding capacity. Traffic between available domains is typically encrypted.

[0043] In certain embodiments, regions are grouped into realms. A realm is a logical collection of regions. Realms are isolated from each other and do not share any data. Regions within the same realm can communicate with each other, but regions in different realms cannot. A CSP customer's tenancy or account may reside in a single realm and span one or more regions belonging to that single realm. Typically, when a customer subscribes to an IaaS service, their tenancy or account is created in a customer-designated region within a realm (referred to as the "home" region). The customer can extend their tenancy to one or more other regions within the realm. A customer cannot access regions that do not reside in the realm in which their tenancy resides.

[0044] An IaaS provider can offer multiple realms, each corresponding to a specific set of customers or users. For example, a commercial realm may be offered for commercial customers. Another example is that a realm may be offered for a specific country or for customers in that country. Yet another example is that a government realm may be offered for a government, for example. For example, a government realm may be created for a specific government and may have a higher security level than a commercial realm. For example, Oracle Cloud Infrastructure (OCI) currently offers a realm for the commercial domain and two realms for the government cloud domain (e.g., FedRAMP accreditation and IL5 accreditation).

[0045] In certain embodiments, AD can be subdivided into one or more fault domains. A fault domain is used to provide anti-affinity. A fault domain is a grouping of infrastructure resources within a single AD. A fault domain allows for the distribution of compute instances so that they are not located on the same physical hardware within a single AD. This is known as anti-affinity. A fault domain refers to a collection of hardware components (computers, switches, etc.) that share a single point of failure. A compute pool is logically divided into fault domains. Therefore, a hardware failure or compute hardware maintenance event affecting one fault domain does not affect instances in other fault domains. Depending on the embodiment, the number of fault domains in each AD may vary. For example, in a particular embodiment, each AD may contain three fault domains. A fault domain functions as a logical data center within the AD.

[0046] When a customer subscribes to an IaaS service, resources from CSPI are provisioned to the customer and associated with the customer's tenancy. The customer can use these provisioned resources to build private networks and deploy resources on these networks. Customer networks hosted in the cloud by CSPI are called Virtual Cloud Networks (VCNs). A customer can configure one or more Virtual Cloud Networks (VCNs) using CSPI resources allocated to them. A VCN is a virtual or software-defined private network. Customer resources deployed on the customer's VCN are compute instances ( For example, this may include virtual machines, bare metal instances, and other resources. These compute instances may represent various customer workloads such as applications, load balancers, and databases. Compute instances deployed on a VCN can communicate with publicly accessible endpoints (public endpoints) via public networks such as the internet, with other instances within the same VCN or other VCNs (e.g., other VCNs of the customer, or VCNs not belonging to the customer), with the customer's on-premises data center or network, with service endpoints, and with other types of endpoints.

[0047] A CSP can offer a variety of services using a CSPI. In some cases, the CSPI customer itself can function like a service provider, offering services using CSPI resources. A service provider can expose service endpoints characterized by identifying information (e.g., IP address, DNS name, and port). A customer's resources (e.g., compute instances) can consume a particular service by accessing the service endpoint for that specific service exposed by the service. These service endpoints are generally endpoints that can be publicly accessed by users over a public communication network such as the internet using the public IP address associated with the endpoint. Publicly accessible network endpoints are sometimes called public endpoints.

[0048] In certain embodiments, a service provider may expose a service through an endpoint of the service (sometimes called a service endpoint). Customers of the service can access the service using this service endpoint. In certain embodiments, the service endpoint provided for a service may be accessible to multiple customers who wish to consume that service. In other implementations, a dedicated service endpoint may be provided to a customer so that only that customer can access the service using that dedicated service endpoint.

[0049] In certain embodiments, a VCN, once created, is associated with a private overlay classless inter-domain routing (CIDR) address space, which is a private overlay IP address range (e.g., 10.0 / 16) assigned to that VCN. A VCN includes associated subnets, route tables, and gateways. While a VCN resides within a single region, it can extend to one or more or all available domains within a region. A gateway is a virtual interface configured for a VCN, enabling traffic communication between the VCN and one or more endpoints outside the VCN. By configuring one or more different types of gateways for a VCN, communication between different types of endpoints can be enabled.

[0050] A VCN may be subdivided into one or more subnets, such as one or more subnets. Thus, a subnet is a constituent unit or partition that can be created within a VCN. A VCN may have one or more subnets. Each subnet within a VCN does not overlap with other subnets within that VCN and is associated with a contiguous range of overlay IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24) that represent an address space subset of the VCN's address space.

[0051] Each compute instance is associated with a virtual network interface card (VNIC). A VNIC is attached. This allows each compute instance to join a subnet in the VCN. A VNIC is a logical representation of a physical network interface card (NIC). Generally, a VNIC is the interface between an entity (e.g., compute instance, service) and a virtual network. A VNIC resides in a subnet and has one or more associated IP addresses and associated security rules or policies. A VNIC is equivalent to a Layer 2 port on a switch. A VNIC connects a compute instance to a subnet in the VCN. The VNIC associated with a compute instance enables the compute instance to be part of a subnet in the VCN and allows the compute instance to communicate (e.g., send and receive packets) with endpoints on the same subnet as the compute instance, endpoints in different subnets within the VCN, or endpoints outside the VCN. Thus, the VNIC associated with a compute instance determines how the compute instance connects to internal and external endpoints in the VCN. The VNIC for a compute instance is created and associated with that compute instance when the compute instance is created and added to a subnet in the VCN. If a subnet consists of a set of compute instances, it includes the VNICs corresponding to the set of compute instances, and each VNIC is connected to a compute instance within the set of computer instances.

[0052] Each compute instance is assigned a private overlay IP address via the VNIC associated with it. This private overlay IP address is assigned to the VNIC associated with the compute instance when the compute instance is created and is used to route the compute instance's traffic. All VNICs within a given subnet use the same route table, security lists, and DHCP options. As described above, each subnet within a VCN does not overlap with other subnets within that VCN and is associated with a contiguous range of overlay IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24) that represents an address space subset of the VCN's address space. For a VNIC on a particular subnet of a VCN, the private overlay IP address assigned to the VNIC is an address from the contiguous range of overlay IP addresses assigned to the subnet.

[0053] In certain embodiments, a compute instance may optionally be assigned additional overlay IP addresses in addition to its private overlay IP address, for example, one or more public IP addresses in the case of a public subnet. These multiple addresses may be assigned to the same VNIC or to multiple VNICs associated with the compute instance. However, each instance has a primary VNIC that is created at instance launch and associated with the overlay private IP address assigned to the instance. This primary VNIC cannot be deleted. Additional VNICs, called secondary VNICs, can be added to an existing instance in the same available domain as the primary VNIC. All VNICs are in the same available domain as the instance. Secondary VNICs may be in the same VCN subnet as the primary VNIC, or they may be in the same VCN or different VCN subnets.

[0054] If a compute instance is in a public subnet, it may optionally be assigned a public IP address. When creating a subnet, you can specify that subnet as either a public or private subnet. A private subnet means that resources within that subnet (e.g., compute instances) and associated VNICs cannot have public overlay IP addresses. A public subnet means that resources within that subnet and associated VNICs can have public IP addresses. Customers can specify their region or You can specify a subnet that exists across a single available domain or multiple available domains within a realm.

[0055] As described above, a VCN may be subdivided into one or more subnets. In certain embodiments, a virtual router (referred to as a VCN VR or simply a VR) configured for a VCN enables communication between subnets within the VCN. For subnets within a VCN, the VR represents a logical gateway for that subnet that enables communication between the subnet (i.e., compute instances on that subnet) and endpoints on other subnets within the VCN and other endpoints outside the VCN. A VCN VR is a logical entity configured to route traffic between VNICs within a VCN and virtual gateways (gateways) associated with the VCN. Gateways are described further below with reference to Figure 1. A VCN VR is a Layer 3 / IP layer concept. In one embodiment, there is one VCN VR for one VCN. This VCN VR potentially has an unlimited number of ports addressed by IP addresses, with one port for each subnet of the VCN. Thus, the VCN VR has a different IP address for each subnet of the VCN to which the VCN VR is connected. The VR is also connected to various gateways configured for the VCN. In certain embodiments, specific overlay IP addresses from a subnet's overlay IP address range are held on ports of the VCN VR for that subnet. For example, consider a VCN having two subnets, each having the associated address ranges 10.0 / 16 and 10.1 / 16. For the first subnet in the VCN having the address range 10.0 / 16, addresses from this range are held on ports of the VCN VR for that subnet. In some cases, the first IP address from this range may be held on the VCN VR. For example, for a subnet having the overlay IP address range 10.0 / 16, the IP address 10.0.0.1 may be held on ports of the VCN VR for that subnet. For the second subnet in the same VCN having the address range 10.1 / 16, the VCN VR may have ports for the second subnet having the IP address 10.1.0.1.A VCN VR has a different IP address for each subnet within the VCN.

[0056] In some other embodiments, each subnet in a VCN may have a VR associated with itself that is addressable by the subnet using a reserved or default IP address associated with the VR. The reserved or default IP address may be, for example, a first IP address from a range of IP addresses associated with that subnet. A VNIC in a subnet can use this default or reserved IP address to communicate with the VR associated with the subnet (e.g., send and receive packets). In such embodiments, the VR is the entry / exit point for that subnet. A VR associated with a subnet in a VCN can communicate with other VRs associated with other subnets in the VCN. A VR can also communicate with gateways associated with the VCN. The VR functionality of a subnet is performed on, or by, one or more NVDs that perform the VNIC functionality of the VNICs in the subnet.

[0057] Route tables, security rules, and DHCP options may be configured for the VCN. The route table is the VCN's virtual route table and contains rules for routing traffic from subnets inside the VCN to destinations outside the VCN, via a gateway or specially configured instance. The VCN's route table can be used to control how packets are forwarded / routed to and from the VCN. It can be customized. DHCP options refer to configuration information that is automatically provided to an instance when the instance is launched.

[0058] Security rules configured for a VCN represent the VCN's overlay firewall rules. Security rules may include ingress and egress rules and may specify the types of traffic allowed to enter and exit instances within the VCN (e.g., based on protocol and port). Customers can choose whether certain rules are stateful or stateless. For example, a customer can allow incoming SSH traffic to a pair of instances from any location by configuring a stateful ingress rule with source CIDR 0.0.0.0 / 0 and destination TCP port 22. Security rules may be implemented using network security groups or security lists. A network security group consists of a set of security rules that apply only to resources within that group. A security list, on the other hand, contains rules that apply to all resources within a subnet that uses that security list. A VCN may include a default security list along with default security rules. DHCP options configured for a VCN provide configuration information that is automatically provided when instances within the VCN start up.

[0059] In certain embodiments, VCN configuration information is determined and stored by the VCN control plane. VCN configuration information may include, for example, address ranges associated with the VCN, subnets and associated information within the VCN, one or more VRs associated with the VCN, compute instances and associated VNICs within the VCN, NVDs that perform various virtualized network functions associated with the VCN (e.g., VNICs, VRs, gateways), VCN status information, and other VCN-related information. In certain embodiments, the VCN distribution service exposes the configuration information or a portion thereof stored by the VCN control plane to the NVD. Using the distributed information, packets can be forwarded to compute instances within the VCN by updating information stored and used by the NVD (e.g., forwarding tables, routing tables, etc.).

[0060] In certain embodiments, the creation of VCNs and subnets is handled by the VCN control plane (CP), and the startup of compute instances is handled by the compute control plane. The compute control plane is configured to allocate physical resources for compute instances and then call the VCN control plane to create VNICs and attach them to compute instances. The VCN CP also sends VCN data mappings to the VCN data plane, which is configured to perform packet forwarding and routing functions. In certain embodiments, the VCN CP provides a distribution service that plays a role in providing updates to the VCN data plane. Examples of VCN control planes are shown in Figures 12, 13, 14, and 15 (see reference numbers 1216, 1316, 1416, and 1516) and are described below.

[0061] A customer may create one or more VCNs using resources hosted by CSPI. Compute instances deployed on a customer VCN may communicate with various endpoints. These endpoints may include endpoints hosted by CSPI and endpoints outside of CSPL.

[0062] Various different architectures for implementing cloud-based services using CSPI are shown in Figures 1, 2, 3, 4, 5, 12, 13, 14, and 16, and are described below. Figure 1 is a high-level diagram of a distributed environment 100 showing an overlay VCN or customer VCN hosted by CSPI according to a particular embodiment. The distributed environment shown in Figure 1 includes multiple components within the overlay network. The distributed environment 100 shown in Figure 1 is merely an example and does not unduly limit the scope of the claimed embodiments. This is not intended to be the case. Many variations, alternatives, and modifications are possible. For example, in some implementations, the distributed environment shown in Figure 1 may have more or fewer systems or components than those shown in Figure 1, may be a combination of two or more systems, or may have different system configurations or arrangements.

[0063] As shown in the example in Figure 1, the distributed environment 100 includes a CSPI 101 that provides services and resources that customers can subscribe to and use to build a virtual cloud network (VCN). In a particular embodiment, the CSPI 101 provides IaaS services to subscriber customers. The data centers within the CSPI 101 may be organized into one or more regions. Figure 1 shows an example of a region, "Region US" 102. The customer has configured a customer VCN 104 for Region 102. The customer may deploy various compute instances on the VCN 104, which may include virtual machines or bare metal instances. Examples of instances include applications, databases, load balancers, etc.

[0064] In the embodiment shown in Figure 1, customer VCN 104 includes two subnets, namely "subnet-1" and "subnet-2", each subnet having its own CIDR IP address range. In Figure 1, the overlay IP address range for subnet-1 is 10.0 / 16, and the address range for subnet-2 is 10.1 / 16. The VCN virtual router 105 represents the logical gateway of the VCN, enabling communication between subnets of VCN 104 and communication with other endpoints outside the VCN. The VCN VR 105 is configured to route traffic between VNICs within VCN 104 and gateways associated with VCN 104. The VCN VR 105 provides ports to each subnet of VCN 104. For example, the VR 105 may provide a port with IP address 10.0.0.1 to subnet-1 and a port with IP address 10.1.0.1 to subnet-2.

[0065] Multiple compute instances may be deployed on each subnet. In this case, compute instances may be virtual machine instances and / or bare metal instances. Compute instances within a subnet may be hosted by one or more host machines within CSPI101. Compute instances join the subnet via the VNIC associated with them. For example, as shown in Figure 1, compute instance C1 is part of subnet-1 via the VNIC associated with it. Similarly, compute instance C2 is part of subnet-1 via the VNIC associated with C2. Similarly, multiple compute instances, which may be virtual machine instances or bare metal instances, may be part of subnet-1. Each compute instance is assigned a private overlay IP address and MAC address via the associated VNIC. For example, in Figure 1, compute instance C1 has the overlay IP address 10.0.0.2 and MAC address M1, and compute instance C2 has the private overlay IP address 10.0.0.3 and MAC address M2. Each compute instance in subnet-1, including compute instances C1 and C2, has a default route to VCN VR105 using IP address 10.0.0.1, which is the IP address of the port of VCN VR105 in subnet-1.

[0066] Multiple compute instances, including virtual machine instances and / or bare metal instances, can be deployed in subnet-2. For example, as shown in Figure 1, compute instances Dl and D2 are part of subnet-2 via the VNIC associated with each compute instance. In the embodiment shown in Figure 1, compute instance D1 has the overlay IP address 10.1.0.2 and MAC address MM1, and compute instance D2 has the private overlay IP address 10.1.0.3 and It has MAC address MM2. Each compute instance in subnet-2, including compute instances D1 and D2, has a default route to VCN VR105 using IP address 10.1.0.1, which is the IP address of the port of VCN VR105 in subnet-2.

[0067] Furthermore, VCN A104 may include one or more load balancers. For example, a load balancer may be provided for a subnet and configured to load balance traffic among multiple compute instances on the subnet. Alternatively, a load balancer may be provided to load balance traffic among subnets within the VCN.

[0068] A specific compute instance deployed on VCN104 can communicate with various different endpoints. These endpoints may include endpoints hosted by CSPI200 and endpoints outside of CSPI200. Endpoints hosted by CSPI101 may include endpoints on the same subnet as a particular compute instance (e.g., communication between two compute instances in subnet-1), endpoints in different subnets but within the same VCN (e.g., communication between a compute instance in subnet-1 and a compute instance in subnet-2), endpoints in different VCNs in the same region (e.g., communication between a compute instance in subnet-1 and an endpoint in a VCN in the same region 106 or 110, or between a compute instance in subnet-1 and an endpoint in service network 110 in the same region), or endpoints in VCNs in different regions (e.g., communication between a compute instance in subnet-1 and an endpoint in a VCN in a different region 108). Furthermore, compute instances in subnets hosted by CSPI101 may communicate with endpoints not hosted by CSPI101 (i.e., outside of CSPI101). These external endpoints include endpoints within the customer's on-premises network 116, endpoints within other remote cloud-hosted networks 118, public endpoints 114 accessible via public networks such as the internet, and other endpoints.

[0069] Communication between compute instances on the same subnet is facilitated using VNICs associated with the source and destination compute instances. For example, compute instance C1 in subnet-1 may want to send a packet to compute instance C2, also in subnet-1. For a packet originating from the source compute instance and destined for another compute instance on the same subnet, this packet is first processed by the VNIC associated with the source compute instance. Processing performed by the VNIC associated with the source compute instance may include determining the packet's destination information from the packet header, identifying any policies (e.g., security lists) configured for the VNIC associated with the source compute instance, determining the packet's next hop, performing any packet encapsulation / decapsulation functions as needed, and forwarding / routing the packet to the next hop to facilitate communication to its intended destination. If the destination compute instance is on the same subnet as the source compute instance, the VNIC associated with the source compute instance is configured to identify the VNIC associated with the destination compute instance and forward packets to be processed to that VNIC. Next, the VNIC associated with the destination compute instance is executed and forwards the packet to the destination compute instance.

[0070] When a compute instance in a subnet communicates packets to an endpoint in a different subnet of the same VCN, the communication is between the source compute instance and the destination compute instance. The process is facilitated by the VNIC associated with each subnet and the VCN VR. For example, if compute instance C1 in subnet-1 in Figure 1 wants to send a packet to compute instance D1 in subnet-2, the packet is first processed by the VNIC associated with compute instance C1. The VNIC associated with compute instance C1 is configured to route the packet to VCN VR105 using the VCN VR's default route or port 10.0.0.1. VCN VR105 is configured to route the packet to subnet-2 using port 10.1.0.1. The packet is then received and processed by the VNIC associated with D1, and the VNIC forwards the packet to compute instance D1.

[0071] To transmit packets from compute instances within VCN104 to endpoints outside VCN104, communication is facilitated by a VNIC associated with the source compute instance, VCN VR105, and a gateway associated with VCN104. One or more types of gateways may be associated with VCN104. A gateway is an interface between the VCN and another endpoint, which is outside the VCN. A gateway is a Layer 3 / IP layer concept that enables communication between the VCN and endpoints outside the VCN. Thus, gateways facilitate traffic flow between the VCN and other VCNs or networks. Various different types of gateways may be configured for the VCN to facilitate different types of communication with different types of endpoints. Communication may take place over a public network (e.g., the Internet) or a private network, depending on the gateway. Various communication protocols may be used for these communications.

[0072] For example, compute instance C1 may want to communicate with an endpoint outside of VCN104. The packet may first be processed by the VNIC associated with source compute instance C1. VNIC processing determines that the packet's destination is outside subnet-1 of Cl. The VNIC associated with C1 may forward the packet to VCN VR105 of VCN104. VCN VR105 then processes the packet and, as part of the processing, determines a specific gateway associated with VCN104 as the packet's next hop based on the packet's destination. VCN VR105 may then forward the packet to a specific identified gateway. For example, if the destination is an endpoint within the customer's operation network, the packet may be forwarded by VCN VR105 to a Dynamic Routing Gateway (DRG) 122 configured for VCN104. The packet then moves from the gateway to the next hop. The ability to forward packets over several hops can facilitate the communication of packets to their intended final destination.

[0073] Various different types of gateways can be configured for a VCN. Examples of gateways that can be configured for a VCN are shown in Figure 1 and described below. Examples of gateways associated with a VCN are also shown in Figures 12, 13, 14, and 15 (for example, gateways shown by reference numbers 1234, 1236, 1238, 1334, 1336, 1338, 1434, 1436, 1438, 1534, 1536, and 1538) and described below. As shown in the embodiment shown in Figure 1, a dynamic routing gateway (DRG) 122 may be added to or associated with a customer VCN 104. The DRG 122 provides a path for private network traffic communication between the customer VCN 104 and another endpoint. The other endpoint may be the customer's on-premises network 116, a VCN 108 in a different region of CSPI 101, or another remote cloud network 118 not hosted by CSPI 101. The customer on-premises network 116 may be a customer network or customer data center built using customer resources. Access to network 116 is generally strictly restricted. For customers who have both a customer on-premises network 116 and one or more VCNs 104 deployed or hosted in the cloud by CSPI 101, the customer may want the on-premises network 116 and the cloud-based VCNs 104 to be able to communicate with each other. This allows the customer to build an extended hybrid environment including the customer's VCNs 104 hosted by CSPI 101 and the on-premises network 116. DRG 122 enables such communication. To enable such communication, a communication channel 124 is configured. In this case, one endpoint of the communication channel is on the customer on-premises network 116, and the other endpoint is on CSPI 101 and connected to the customer VCN 104. The communication channel 124 can be via a public communication network such as the internet, or a private communication network. Various different communication protocols can be used, such as IPsec VPN technology on a public communication network such as the internet, or Oracle®'s FastConnect technology which uses a private network instead of a public network. Devices or equipment within the customer on-premises network 116 that form two endpoints are called customer premise equipment (CPE), such as CPE126 shown in Figure 1. The endpoint on the CSPI101 side may be the host machine running DRG122.

[0074] In certain embodiments, Remote Peering Connection (R A PC can be added to the DRG. This allows a customer to peer one VCN with another VCN in a different region. Using such RPC, customer VCN104 can connect to VCN108 in a different region using DRG122. DRG122 can also connect to other remote cloud networks 118 not hosted by CSPI101, such as Microsoft® Azure Cloud. Amazon® may be used to communicate with AWS Cloud and other services.

[0075] As shown in Figure 1, an Internet Gateway (IGW) 120 may be configured for the customer VCN 104 to enable compute instances on the customer VCN 104 to communicate with a public endpoint 114 accessible via a public network such as the Internet. The IGW 120 is a gateway that connects the VCN to a public network such as the Internet. The IGW 120 enables public subnets within a VCN, such as VCN 104 (resources within the public subnet have public overlay IP addresses), to directly access the public endpoint 112 on the public network 114, such as the Internet. Connections can be initiated from subnets within VCN 104 or from the Internet using the IGW 120.

[0076] A Network Address Translation (NAT) gateway 128 can be configured for the customer's VCN104. The NAT gateway 128 This enables cloud resources within a customer's VCN that do not have a dedicated public overlay IP address to access the internet without directly exposing them to incoming internet connections (e.g., L4-L7 connections). This allows private subnets within a VCN, such as private subnet-1 of VCN104, to privately access public endpoints on the internet. The NAT gateway can initiate connections from private subnets to the public internet, but it cannot initiate connections from the internet to private subnets.

[0077] In a particular embodiment, a service gateway for customer VCN104 (Service Gateway (SGW)126 can be configured. SGW126 is connected to VCN104 and The service network 110 provides a path for private network traffic to and from supported service endpoints. In certain embodiments, the service network 110 may be provided by a CSP and may provide a variety of services. An example of such a service network is the Oracle® service network, which provides a variety of services that customers can use. For example, compute instances (e.g., database systems) in a private subnet of a customer VCN 104 can back up data to a service endpoint (e.g., object storage) without requiring a public IP address or internet access. In some embodiments, a VCN may only have one SGW, and connections can only be initiated from subnets within the VCN, but not from the service network 110. When peering a VCN with another VCN, resources in the other VCN typically cannot access the SGW. Resources in an on-premises network connected to a VCN via FastConnect or VPN Connect can also use the service gateway configured for that VCN.

[0078] In some implementations, SGW126 uses service-classless inter-domain routing (CIDR) labels. A CIDR label is a string representing all regional public IP address ranges for a service or group of services of interest. Customers use service CIDR labels to control traffic to services when configuring SGW and associated routing rules. Customers can optionally use service CIDR labels when configuring security rules without having to adjust the security rules if the public IP addresses of services change in the future.

[0079] Local Peering Gateway (LPG) 132 is for customers This is a gateway that can be added to VCN104, enabling VCN104 to peer with other VCNs within the same region. Peering means that VCNs communicate using private IP addresses without traffic traversing a public network such as the internet or routing traffic through the customer's on-premises network 116. In a preferred embodiment, the VCN has a separate LPG for each established peering. Local peering, or VCN peering, is a common practice used to establish network connectivity between different applications or infrastructure management functions.

[0080] Service providers, such as service providers on service network 110, may provide access to their services using different access models. According to the public access model, a service may be exposed as a public endpoint, publicly accessible by compute instances within the customer VCN via a public network such as the internet, or it may be privately accessible via SGW126. According to a specific private access model, a service may be accessible as a private IP endpoint within a private subnet within the customer VCN. This is called Private Endpoint (PE) access and allows service providers to expose their services as instances within the customer's private network. A private endpoint resource represents a service within the customer VCN. Each PE appears as a VNIC (called a PE-VNIC, having one or more private IPs) selected by the customer from a subnet within the customer VCN. The PE thus provides a way to provide services within the customer's private VCN subnet using VNICs. Because the endpoint is exposed as a VNIC, the PE VNIC has all the characteristics associated with a VNIC, such as routing rules and Security lists and other similar tools can be used.

[0081] Service providers enable access via PE by registering their services. Providers can associate policies with services that regulate their visibility to customer tenants. Providers can register multiple services under a single virtual IP address (VIP), especially in the case of multi-tenant services. Multiple private endpoints may exist representing the same service (across multiple VCNs).

[0082] Subsequently, compute instances within the private subnet can access the service using the PE VNIC's private IP address or service DNS name. Compute instances within the customer VCN can access the service by sending traffic to the PE's private IP address within the customer VCN. The Private Access Gateway (PAGW) 130 is a gateway resource that can connect to a service provider VCN (e.g., a VCN within service network 110) and act as the entry / exit point for all traffic to and from the customer subnet private endpoint. The PAGW 130 allows the provider to scale the number of PE connections without utilizing internal IP address resources. The provider only needs to configure one PAGW for any number of services registered in a single VCN. The provider can present a service as a private endpoint in multiple VCNs for one or more customers. From the customer's perspective, the PE VNIC appears to be connected to the service the customer wants to interact with, rather than to the customer's instances. Traffic directed to the private endpoint is routed to the service via the PAGW 130. These are called customer-to-service private connections (C2S connections).

[0083] Furthermore, using the PE concept, traffic is transmitted over FastConnect / IPsec links and customer VCNs. By allowing traffic to flow through private endpoints within the network, private access for the service can also be extended to the customer's on-premises network and data center. Furthermore, by allowing traffic to flow between LPG132 and the PE within the customer's VCN, private access for the service can also be extended to the customer's peered VCN.

[0084] Customers can control VCN routing at the subnet level, allowing them to specify which subnets use which gateways within their VCN, such as VCN104. The VCN's route table can be used to determine whether traffic can be routed outside the VCN through a particular gateway. For example, in a specific case, the route table for a public subnet within customer VCN104 might send non-local traffic via IGW120. The route table for a private subnet within the same customer VCN104 might send traffic to a CSP service via SGW126. All remaining traffic might be sent via NAT gateway 128. The route table controls only traffic leaving the VCN.

[0085] Security lists associated with a VCN are used to control traffic entering the VCN through the gateway via inbound connections. All resources within a subnet use the same route table and security lists. Security lists may be used to control specific types of traffic entering and leaving instances within a VCN subnet. Security list rules may include inbound rules and outbound rules. For example, inbound rules may allow traffic that The source address range may be specified, and the exit rule may specify the allowed destination address range. Security rules may specify specific protocols (e.g., TCP, ICMP), specific ports (e.g., 22 for SSH, 3389 for Windows® RDP), etc. In certain implementations, the instance's operating system may enforce its own firewall rules that match the security list rules. Rules may be stateful (e.g., connections are tracked and responses are automatically allowed without explicit security list rules for response traffic) or stateless.

[0086] Access from a customer VCN (i.e., resources or compute instances deployed on VCN104) can be classified as public access, private access, or dedicated access. Public access refers to an access model for accessing public endpoints using public IP addresses or NAT. Private access allows customer workloads within VCN104 with private IP addresses (e.g., resources in a private subnet) to access services without traversing a public network such as the internet. In certain embodiments, CSPI101 allows customer VCN workloads with private IP addresses to access services (or their public service endpoints) using a service gateway. Thus, the service gateway provides a private access model by establishing a virtual link between the customer VCN and the public endpoint of the service that resides outside the customer's private network.

[0087] Furthermore, CSPI uses dedicated platforms that utilize technologies such as FastConnect public peering. It can provide brick access. In this case, the customer's on-premises instance can use FastConnect connectivity without going through a public network such as the internet. A customer can access one or more services within the VCN. Additionally, CSPI may provide dedicated private access using FastConnect private peering. In this case, customer on-premises instances with private IP addresses can access customer VCN workloads using FastConnect connectivity. FastConnect is a network connection that replaces connecting customer on-premises networks to CSPI and its services using the public internet. It provides an easy, flexible, and economical way to create dedicated private connections with higher bandwidth options and a reliable, consistent networking experience compared to network-based connections.

[0088] Figure 1 and the above description illustrate the various virtualization components in an exemplary virtual network. As mentioned above, the virtual network is built on an underlying physical network or infrastructure network. Figure 2 is a simplified architecture diagram showing the physical components within the physical network in the CSPI200 that provide the foundation for the virtual network, according to a particular embodiment. As shown, the CSPI200 provides a distributed environment including components and resources (e.g., compute resources, memory resources, and networking resources) provided by a Cloud Service Provider (CSP). These components and resources are used to provide cloud services (e.g., IaaS services) to subscribers, i.e., customers who subscribe to one or more services provided by the CSP. Based on the services the customer subscribes to, the CSPI200 provides some resources (e.g., compute resources, memory resources, and networking resources) to the customer. The customer then uses the physical compute resources, memory resources, and networking resources provided by the CSPI200 to create their own cloud-based (i.e., CSPI-hosted) customizations. It is possible to build a private virtual network. As mentioned earlier, these customer networks are called virtual cloud networks (VCNs). Customers can deploy one or more customer resources, such as compute instances, to these customer VCNs. Compute instances may be virtual machines, bare metal instances, etc. CSPI200 provides infrastructure and a set of complementary cloud services that enable customers to build and run a wide range of applications and services in a highly available host environment.

[0089] In the exemplary embodiment shown in Figure 2, the physical components of the CSPI200 include one or more physical host machines or physical servers (e.g., 202, 206, 208), network virtualization devices (NVDs) (e.g., 210, 212), top-of-rack (TOR) switches (e.g., 214, 216), a physical network (e.g., 218), and switches within the physical network 218. The physical host machines or servers may host and run various compute instances participating in one or more subnets of the VCN. Compute instances may include virtual machine instances and bare metal instances. For example, the various compute instances shown in Figure 1 may be hosted by the physical host machines shown in Figure 2. Virtual machine compute instances in the VCN may run on one host machine or on several different host machines. The physical host machines may also host virtual host machines, container-based hosts or functions, etc. The VNIC and VCN VR shown in Figure 1 may be run by the NVDs shown in Figure 2. The gateway shown in Figure 1 may be run by the host machine and / or NVD shown in Figure 2.

[0090] A host machine or server may run a hypervisor (also known as a virtual machine monitor or VMM) that creates and enables a virtualized environment on the host machine. Virtualization or a virtualized environment facilitates cloud-based computing. One or more compute instances may be created, run, and managed on the host machine by the hypervisor on the host machine. The hypervisor on the host machine can share the host machine's physical compute resources (e.g., compute resources, memory resources, and networking resources) among the various compute instances running on the host machine.

[0091] For example, as shown in Figure 2, host machines 202 and 208 run hypervisors 260 and 266, respectively. These hypervisors may be implemented using software, firmware, hardware, or a combination thereof. Typically, a hypervisor is a process or software layer residing in the host machine's operating system (OS), which runs on the host machine's hardware processors. A hypervisor provides a virtualization environment that allows the host machine's physical computing resources (e.g., processing resources such as processors / cores, memory resources, and networking resources) to be shared among various virtual machine computing instances running on the host machine. For example, in Figure 2, hypervisor 260 may reside in the OS of host machine 202, allowing the host machine 202's computing resources (e.g., processing resources, memory resources, and networking resources) to be shared among computing instances (e.g., virtual machines) running on host machine 202. A virtual machine can have its own operating system (called a guest operating system). This guest operating system may be the same as or different from the host machine's OS. The operating system of a virtual machine running on a host machine may be the same as, or different from, the operating systems of other virtual machines running on the same host machine. Therefore, the hypervisor can share the same computing resources of the host machine while running multiple virtual machines. The operating systems can run in parallel. The host machines shown in Figure 2 may have the same type of hypervisor or different types of hypervisors.

[0092] Compute instances may be virtual machine instances or bare metal instances. In Figure 2, compute instance 268 on host machine 202 and compute instance 274 on host machine 208 are examples of virtual machine instances. Host machine 206 is an example of a bare metal instance provided to a customer.

[0093] In certain examples, the entire host machine may be provided to a single customer, and one or more compute instances (either virtual machines or bare metal instances) hosted by that host machine may all belong to the same customer. In other examples, the host machine may be shared among multiple customers (i.e., multiple tenants). In such a multi-tenant scenario, the host machine may host virtual machine compute instances belonging to different customers. These compute instances may be members of different VCNs of different customers. In certain embodiments, bare metal compute instances are hosted by bare metal servers without a hypervisor. When bare metal compute instances are provided, a single customer or tenant maintains control of the physical CPU, memory, and network interfaces of the host machine hosting the bare metal instances, and the host machine is not shared with other customers or tenants.

[0094] As mentioned above, each compute instance that is part of a VCN is associated with a VNIC that enables the compute instance to be a member of the VCN's subnet. The VNIC associated with a compute instance facilitates the communication of packets or frames to and from the compute instance. The VNIC is associated with the compute instance when it is created. In certain embodiments, for a compute instance run by a host machine, the VNIC associated with that compute instance is run by an NVD connected to the host machine. For example, in Figure 2, host machine 202 runs virtual machine compute instance 268 associated with VNIC 276, and VNIC 276 is run by an NVD 210 connected to host machine 202. In another example, bare metal instance 272 hosted by host machine 206 is associated with VNIC 280, which is run by an NVD 212 connected to host machine 206. In yet another example, VNIC 284 is associated with compute instance 274 run by host machine 208, and VNIC 284 is run by an NVD 212 connected to host machine 208.

[0095] For compute instances hosted by a host machine, an NVD connected to that host machine executes a VCN VR corresponding to the VCN of which the compute instance is a member. For example, in the embodiment shown in Figure 2, NVD210 executes VCN VR277 corresponding to the VCN of which compute instance 268 is a member. Additionally, NVD212 may execute one or more VCN VR283 corresponding to the VCNs of compute instances hosted by host machines 206 and 208.

[0096] A host machine may include one or more network interface cards (NICs) that enable it to connect to other devices. The NICs on the host machine may provide one or more ports (or interfaces) that enable the host machine to communicate with another device. For example, the host machine may be connected to the NVD using one or more ports (or interfaces) provided on the host machine and the NVD. Alternatively, the host machine may be connected to other devices, such as another host machine.

[0097] For example, in Figure 2, host machine 202 is connected to NVD210 using a link 220 that extends between port 234 provided by NIC 232 of host machine 202 and port 236 of NVD210. Host machine 206 is connected to NVD212 using a link 224 that extends between port 246 provided by NIC 244 of host machine 206 and port 248 of NVD212. Host machine 208 is connected to NVD212 using a link 226 that extends between port 252 provided by NIC 250 of host machine 208 and port 254 of NVD212.

[0098] Similarly, the NVDs are connected via communication links to top-of-rack (TOR) switches connected to a physical network 218 (also called a switch fabric). In certain embodiments, the links between the host machines and the NVDs, and between the NVDs and the TOR switches, are Ethernet® links. For example, in Figure 2, NVDs 210 and 212 are connected to TOR switches 214 and 216, respectively, via links 228 and 230. In certain embodiments, links 220, 224, 226, 228, and 230 are Ethernet® links. The collection of host machines and NVDs connected to the TOR is sometimes referred to as a rack.

[0099] The physical network 218 provides a communication fabric that enables interoperability between TOR switches. The physical network 218 may be a multi-layer network. In a particular implementation, the physical network 218 is a multi-layer Clos network of switches, and TOR switches 214 and 216 represent leaf-level nodes of the multi-layer and multi-node physical switching network 218. Different Clos network configurations are possible, including but not limited to 2-layer, 3-layer, 4-layer, 5-layer networks, and generally "n"-layer networks. An example of a Clos network is shown in Figure 5 and described below.

[0100] Various connection configurations are possible between the host machine and the VD, including one-to-one, many-to-one, and one-to-many configurations. In an example of a one-to-one configuration, each host machine is connected to its own separate NVD. For example, in Figure 2, host machine 202 is connected to NVD210 via host machine 202's NIC232. In a many-to-one configuration, multiple host machines are connected to a single NVD. For example, in Figure 2, host machines 206 and 208 are connected to the same NVD212 via NIC244 and 250, respectively.

[0101] In a one-to-many configuration, one host machine is connected to multiple NVDs. Figure 3 shows an example within CSPI300 where a host machine is connected to multiple NVDs. As shown in Figure 3, the host machine 302 has a network interface card (NIC) 304 which includes multiple ports 306 and 30S. The host machine 300 is connected to the first NVD 310 via port 306 and link 320, and to the second NVD 312 via port 308 and link 322. Ports 306 and 308 may be Ethernet® ports, and links 320 and 322 between the host machine 302 and the NVDs 310 and 312 may be Ethernet® links. The NVD 310 is connected to the first TOR switch 314, and the NVD 312 is connected to the second TOR switch 316. The links between the NVDs 310 and 312 and the TOR switches 314 and 316 may be Ethernet® links. TOR switches 314 and 316 represent layer-0 switching devices within a multilayer physical network 318.

[0102] The configuration shown in Figure 3 provides two separate physical network paths from the physical switch network 318 to the host machine 302: a first path from TOR switch 314 through NVD 310 to the host machine 302, and a second path from TOR switch 316 through NVD 312 to the host machine 302. The separate paths provide enhanced availability (referred to as high availability) for the host machine 302. If there is a problem with one of the paths (e.g., one link in the path fails) or a problem with a device (e.g., a particular NVD is not functioning), the other path may be used for communication with the host machine 302.

[0103] In the configuration shown in Figure 3, the host machine is connected to two different NVDs using two different ports provided by the host machine's NIC. In other embodiments, the host machine may include multiple NICs that enable connections between the host machine and multiple NVDs.

[0104] Referring again to Figure 2, the NVD is a physical device or component that performs one or more network virtualization and / or storage virtualization functions. The NVD can be any device having one or more processing units (e.g., CPU, network processing unit (NPU), FPGA, packet processing pipeline), memory including a cache, and ports. Various virtualization functions may be performed by software / firmware executed by one or more processing units of the NVD.

[0105] NVDs may be implemented in various different forms. For example, in certain embodiments, an NVD may be implemented as an interface card called a smart NIC or intelligent NIC with an integrated processor. A smart NIC is a separate device from the NIC on the host machine. In Figure 2, NVD210 may be implemented as a smart NIC connected to host machine 202, and NVD212 may be implemented as smart NICs connected to host machines 206 and 208.

[0106] However, the smart NIC is just one example of an NVD implementation. Various other implementations are possible. For example, in some other implementations, the NVD or one or more functions performed by the NVD may be incorporated into or performed by one or more host machines, one or more TOR switches, and other components of the CSPI200. For example, the NVD may be integrated into the host machine. In this case, the functions performed by the NVD are performed by the host machine. As another example, the NVD may be part of a TOR switch, or the TOR switch may be configured to perform functions performed by the NVD, enabling the TOR switch to perform various complex packet translations used in public clouds. A TOR that performs the functions of the NVD is sometimes called a smart TOR. In yet another implementation that provides customers with virtual machine (VM) instances rather than bare metal (BM) instances, the functions provided by the NVD may be implemented within the hypervisor of the host machine. In some other implementations, some of the functions of the NVD may be offloaded to a centralized service running on a set of host machines.

[0107] As shown in Figure 2, in certain embodiments, such as when implemented as a smart NIC, the NVD may have multiple physical ports that enable it to be connected to one or more host machines and one or more TOR switches. The ports on the NVD can be classified as host-facing ports (also called "south ports") or network-facing or TOR-facing ports (also called "north ports"). The host-facing ports of the NVD are used to connect the NVD to a host machine. These are the ports used. In Figure 2, examples of host-facing ports include port 236 on the NVD210 and ports 248 and 254 on the NVD212. The network-facing ports of the NVD are the ports used to connect the NVD to the TOR switch. Examples of network-facing ports in Figure 2 include port 256 on the NVD210 and port 258 on the NVD212. As shown in Figure 2, the NVD210 is connected to the TOR switch 214 using a link 228 that extends from port 256 on the NVD210 to the TOR switch 214. Similarly, the NVD212 is connected to the TOR switch 216 using a link 230 that extends from port 258 on the NVD212 to the TOR switch 216.

[0108] The NVD may receive packets and frames from the host machine (e.g., packets and frames generated by compute instances hosted by the host machine) via its host-facing port, perform the necessary packet processing, and then forward the packets and frames to the TOR switch via its network-facing port. The NVD may also receive packets and frames from the TOR switch via its network-facing port, perform the necessary packet processing, and then forward the packets and frames to the host machine via its host-facing port.

[0109] In certain embodiments, multiple ports and associated links may be provided between the NVD and the TOR switch. By aggregating these ports and links, a link aggregator group (LAG) of multiple ports or links can be formed. A LAG (Loop Aggregation) can be formed. Link aggregation allows multiple physical links between two endpoints (e.g., between an NVD and a TOR switch) to be treated as a single logical link. All physical links within a given LAG can operate in full-duplex mode at the same speed. LAGs help increase the bandwidth and reliability of the connection between two endpoints. If one of the physical links in the LAG fails, traffic is dynamically and transparently reassigned to another physical link within the LAG. Aggregated physical links provide higher bandwidth than individual links. Multiple ports associated with a LAG are treated as a single logical port. Traffic can be load-balanced across multiple physical links in the LAG. One or more LAGs may be configured between two endpoints. The two endpoints may be, for example, between an NVD and a TOR switch, or between a host machine and an NVD.

[0110] NVD implements or performs network virtualization functions. These functions are performed by software / firmware run by NVD. Examples of network virtualization functions include, but are not limited to, packet encapsulation and decapsulation functions, functions for creating VCN networks, functions for implementing network policies such as VCN security list (firewall) functions, and functions for facilitating the routing and forwarding of packets to and from compute instances within the VCN. In certain embodiments, upon receiving a packet, NVD is configured to run a packet processing pipeline to process the packet and determine how to forward or route it. As part of this packet processing pipeline, NVD provides the execution of VNICs related to compute instances within the VCN, the execution of virtual routers (VRs) related to the VCN, packet encapsulation and decapsulation to facilitate forwarding or routing within the virtual network, the execution of specific gateways (e.g., local peering gateways), the implementation of security lists, network security groups, network address translation (NAT) functions (e.g., translation from public IP to private IP on a per-host basis), throttling functions, and other functions.

[0111] In some embodiments, the packet processing data path in NVD may include multiple packet pipelines. Each packet pipeline is a series of packet translations. It consists of stages. In some implementations, upon receiving a packet, it is parsed and classified into a single pipeline. The packet is then processed linearly stage by stage until it is discarded or sent out via the NVD interface. These stages provide the basic functional packet processing building blocks (e.g., header validation, throttling, insertion of new Layer 2 headers, L4 firewall execution, VCN encapsulation / decapsulation), and as a result, new pipelines can be built by assembling existing stages, and new functionality can be added by creating new stages and inserting them into existing pipelines.

[0112] The NVD can perform both control plane and data plane functions corresponding to the VCN's control plane and data plane. Examples of the VCN control plane are shown in Figures 12, 13, 14, and 15 (see references 1216, 1316, 1416, and 1516) and are described below. Examples of the VCN data plane are shown in Figures 12, 13, 14, and 15 (see references 1218, 1318, 1418, and 1518) and are described below. Control plane functions include functions used to configure the network that controls how data is forwarded (e.g., setting routes and route tables, configuring VNICs). In certain embodiments, a VCN control plane is provided that centrally calculates the mapping of all overlays to the substrate and exposes it to the NVD and virtual network edge devices (e.g., various gateways such as DRG, SGW, IGW). Firewall rules may also be exposed using the same mechanism. In certain embodiments, the NVD retrieves only the mappings relevant to that NVD. The data plane function includes the ability to perform the actual routing / forwarding of packets based on the configuration set using the control plane. The VCN data plane is implemented by encapsulating customer network packets before they pass through the underlying network. The encapsulation / decapsulation function is implemented on the NVD. In certain embodiments, the NVD is configured to intercept all network packets entering and leaving the host machine and to perform network virtualization functions.

[0113] As described above, NVD performs various virtualization functions, including VNICs and VCN VRs. An NVD may run VNICs associated with compute instances hosted by one or more host machines connected to a VNIC. For example, as shown in Figure 2, NVD210 performs the functions of VNIC276 associated with compute instance 268 hosted by host machine 202 connected to NVD210. As another example, NVD212 runs VNIC280 associated with bare-metal compute instance 272 hosted by host machine 206 and VNIC284 associated with compute instance 274 hosted by host machine 208. A host machine may host compute instances belonging to different VCNs belonging to different customers. An NVD connected to a host machine may run VNICs corresponding to compute instances (i.e., perform VNIC-related functions).

[0114] Furthermore, the NVD runs a VCN virtual router corresponding to the VCN of the compute instance. For example, in the embodiment shown in Figure 2, NVD210 runs VCN VR277 corresponding to the VCN to which compute instance 268 belongs. NVD212 runs one or more VCN VR283 corresponding to one or more VCNs to which compute instances hosted on host machines 206 and 208 belong. In a particular embodiment, a VCN VR corresponding to a VCN is run by all NVDs connected to a host machine that hosts at least one compute instance belonging to that VCN. If a host machine hosts compute instances belonging to a different VCN, the NVDs connected to that host machine may run VCN VRs corresponding to different VCNs.

[0115] In addition to VNICs and VCN VRs, an NVD may include one or more hardware components that run various software (e.g., daemons) and facilitate various network virtualization functions performed by the NVD. For simplicity, these various components are grouped as “packet processing components” as shown in Figure 2. For example, NVD210 includes packet processing component 286, and NVD212 includes packet processing component 288. For example, the NVD’s packet processing components may include a packet processor configured to monitor all packets received and communicated using the NVD and to store network information by interacting with the NVD’s ports and hardware interfaces. Network information may include, for example, network flow information to identify different network flows processed by the NVD and information about each flow (e.g., statistics for each flow). In certain embodiments, network flow information may be stored on a per-VNIC basis. The packet processor may perform per-packet operations and may implement stateful NAT and L4 firewalls (FW). As another example, a packet processing component may include a replication agent configured to replicate information stored by the NVD to one or more different replication target stores. As an example, the packet processing component may include a logging agent configured to perform NVD's logging function. The component may include software to monitor the performance and health of the NVD, and optionally, the status and health of other components connected to the NVD.

[0116] Figure 1 shows components of an exemplary virtual network or overlay network, including a VCN, subnets within the VCN, compute instances deployed on the subnets, VNICs associated with the compute instances, a VR for the VCN, and a set of gateways configured for the VCN. The overlay components shown in Figure 1 may be run or hosted by one or more of the physical components shown in Figure 2. For example, compute instances within a VCN may be run or hosted by one or more host machines shown in Figure 2. In the case of compute instances hosted by a host machine, the VNIC associated with that compute instance is typically run by an NVD connected to that host machine (i.e., the VNIC functionality is provided by the NVD connected to that host machine). The VCN VR functionality of the VCN is run by all NVDs connected to the host machines that host or run the compute instances that are part of that VCN. Gateways associated with the VCN may be run by one or more different types of NVDs. For example, some gateways may be run by smart NICs, and others may be run by one or more host machines or other implementations of NVDs.

[0117] As described above, compute instances within a customer VCN can communicate with various different endpoints. These endpoints may be in the same subnet as the source compute instance, in a different subnet but still within the same VCN, or may include endpoints outside the source compute instance's VCN. These communications are facilitated using the VNIC associated with the compute instance, the VCN VR, and the gateway associated with the VCN.

[0118] Communication between two compute instances on the same subnet within a VCN is facilitated using VNICs associated with the source and destination compute instances. The source and destination compute instances may be hosted by the same host machine or by different host machines. Packets originating from the source compute instance are transmitted from the host machine hosting the source compute instance. The packets may be forwarded to an NVD connected to the host machine. In the NVD, the packets are processed using a packet processing pipeline, which may include the execution of a VNIC associated with the source compute instance. Because the destination endpoint of the packets is on the same subnet, the execution of the VNIC associated with the source compute instance forwards the packets to an NVD running the VNIC associated with the destination compute instance, where the NVD processes the packets and forwards them to the destination compute instance. The VNICs associated with the source and destination compute instances may run on the same NVD (for example, if both the source and destination compute instances are hosted by the same host machine) or on different NVDs (for example, if the source and destination compute instances are hosted by different host machines connected to different NVDs). The VNICs may use routing / forwarding tables stored by the NVD to determine the next hop for the packets.

[0119] When a packet is communicated from a compute instance within a subnet to an endpoint in a different subnet within the same VCN, the packet originating from the source compute instance is communicated from the host machine hosting the source compute instance to the NVD connected to that host machine. In the NVD, the packet is processed using a packet processing pipeline that may include the execution of one or more VNICs and a VR associated with the VCN. For example, as part of the packet processing pipeline, the NVD executes or invokes a function corresponding to the VNIC associated with the source compute instance (also referred to as executing the VNIC). The function executed by the VNIC may include examining the VLAN tag on the packet. Because the packet's destination is outside the subnet, a VCN VR function is invoked and executed by the NVD. The VCN VR then routes the packet to the NVD executing the VNIC associated with the destination compute instance. The VNIC associated with the destination compute instance then processes the packet and forwards it to the destination compute instance. The VNICs associated with the source compute instance and the destination compute instance may run on the same NVD (for example, if both the source compute instance and the destination compute instance are hosted by the same host machine), or they may run on different NVDs (for example, if the source compute instance and the destination compute instance are hosted by different host machines connected to different NVDs).

[0120] If the packet's destination is outside the source compute instance's VCN, the packet originating from the source compute instance is communicated from the host machine hosting the source compute instance to the NVD connected to that host machine. The NVD runs the VNIC associated with the source compute instance. Because the packet's destination endpoint is outside the VCN, the packet is processed by the VCN VR for that VCN. The NVD invokes VCN VR functionality, which may result in the packet being forwarded to an NVD running the appropriate gateway associated with the VCN. For example, if the destination is an endpoint within the customer's on-premises network, the packet may be forwarded by the VCN VR to an NVD running the DRG gateway configured for the VCN. The VCN VR may run on the same NVD running the VNIC associated with the source compute instance, or it may run on a different NVD. The gateway may run on a smart NIC, the host machine, or an NVD which is another NVD implementation example. The packet is then processed by the gateway and forwarded to the next hop to facilitate communication of the packet to its intended destination endpoint. For example, in the embodiment shown in Figure 2, packets originating from compute instance 268 may be communicated from host machine 202 to NVD210 via link 220 (using NIC232). VNIC276 on NVD210 is called because it is the VNIC associated with source compute instance 268. VNIC276 encapsulates the packets. It is configured to inspect the information it receives, determine the next hop for forwarding the packet to facilitate communication of the packet to the intended destination endpoint, and forward the packet to the determined next hop.

[0121] Compute instances deployed on a VCN can communicate with various different endpoints. These endpoints may include endpoints hosted by CSPI200 and endpoints outside of CSPI200. Endpoints hosted by CSPI200 may include instances within the same VCN or other VCNs (which may be the customer's VCN or a VCN not belonging to the customer). Communication between endpoints hosted by CSPI200 may be performed over the physical network 218. Compute instances may also communicate with endpoints that are not hosted by CSPI200 or are outside of CSPI200. Examples of these endpoints include endpoints within the customer's on-premises network or data center, or public endpoints accessible over a public network such as the Internet. Communication with endpoints outside of CSPI200 may be performed over a public network (e.g., the Internet) (not shown in Figure 2) or a private network (not shown in Figure 2) using various communication protocols.

[0122] The architecture of the CSPI200 shown in Figure 2 is merely an example and is not intended to be limiting. Alternative embodiments are possible, and variations, substitutions, and modifications are possible. For example, in some implementations, the CSPI200 may have more or fewer systems or components than those shown in Figure 2, may be a combination of two or more systems, or may have different system configurations or arrangements. The systems, subsystems, and other components shown in Figure 2 may be implemented using software (e.g., code, instructions, programs), hardware, or combinations thereof, executed by one or more processing units (e.g., processors, cores) of each system. The software may be stored in a non-temporary storage medium (e.g., a memory device).

[0123] Figure 4 shows the connection between a host machine and an NVD to provide I / O virtualization to support multi-tenancy, according to a particular embodiment. As shown in Figure 4, the host machine 402 runs a hypervisor 404 that provides the virtualization environment. The host machine 402 runs two virtual machine instances, namely VM1 406 belonging to customer / tenant #1 and VM2 408 belonging to customer / tenant #2. The host machine 402 includes a physical NIC 410 connected to the NVD 412 via link 414. Each compute instance is connected to a VNIC run by the NVD 412. In the embodiment of Figure 4, VM1 406 is connected to VNIC-VM1 420 and VM2 408 is connected to VNIC-VM2 422.

[0124] As shown in Figure 4, NIC410 includes two logical NICs, namely logical NIC A 416 and logical NIC B 418. Each virtual machine is connected to its own logical NIC and configured to operate with its own logical NIC. For example, VM1 406 is connected to logical NIC A 416, and VM2 408 is connected to logical NIC B 418. Although the host machine 402 consists of only one physical NIC 410 shared by multiple tenants, the logical NICs allow each tenant's virtual machine to believe that it owns its own host machine and NIC.

[0125] In a particular embodiment, each logical NIC is assigned its own VLAN ID. Therefore, logical NIC A416 of tenant #1 is assigned a specific VLAN ID. As a result, a different VLAN ID is assigned to logical NIC B 418 of tenant #2. When a packet is communicated from VM1 406, the hypervisor attaches the tag assigned to tenant #1 to the packet and then communicates the packet from host machine 402 to NVD412 via link 414. Similarly, when a packet is communicated from VM2 408, the hypervisor attaches the tag assigned to tenant #2 to the packet and then communicates the packet from host machine 402 to NVD412 via link 414. Therefore, packet 424 communicated from host machine 402 to NVD412 has an associated tag 426 that identifies a specific tenant and associated VM. When packet 424 is received on NVD from host machine 402, the associated tag 426 of the packet is used to determine whether the packet should be processed by VNIC-VM1 420 or VNIC-VM2 422. The packet is then processed by the corresponding VNIC. The configuration shown in Figure 4 allows each tenant's compute instance to believe it owns its own host machine and NIC. The configuration shown in Figure 4 provides I / O virtualization to support multi-tenancy.

[0126] Figure 5 is a schematic block diagram showing a physical network 500 according to a particular embodiment. The embodiment shown in Figure 5 is constructed as a Clos network. A Clos network is a specific type of network topology designed to provide connectivity redundancy while maintaining high bimodal bandwidth and maximum resource utilization. A Clos network is a type of non-blocking, multi-stage or multi-layer switching network, where the number of stages or layers may be 2, 3, 4, 5, etc. The embodiment shown in Figure 5 is a three-layer network including layer 1, layer 2, and layer 3. A TOR switch 504 represents a layer-0 switch in the Clos network. One or more NVDs are connected to the TOR switch. Layer-0 switches are also called edge devices in the physical network. Layer-0 switches are connected to layer-1 switches, also called leaf switches. In the embodiment shown in Figure 5, "n" layer-0 TOR switches are connected to "n" layer-1 switches to form pods. Each layer-0 switch in a pod is interconnected to all layer-1 switches in the pod, but switches between pods are not connected. In a particular implementation, two pods are referred to as a block. Each block is serviced by or connected to n Layer-2 switches (also called spine switches). A physical network topology can have several blocks. Similarly, a Layer-2 switch is connected to n Layer-3 switches (also called superspine switches). Packet communication over the physical network typically uses one or more Layer 3 communication protocols. Typically, all layers of the physical network except the TOR layer are n-way redundant, thus achieving high availability. The physical network can be extended by specifying policies for pods and blocks to control the mutual visibility of switches in the physical network.

[0127] A key feature of Clos networks is that the maximum hop count required to travel from one Layer-0 switch to another (or from an NVD connected to a Layer-0 switch to another NVD connected to a Layer-0 switch) remains constant. For example, in a Layer 3 Clos network, a packet requires a maximum of 7 hops to travel from one NVD to another. In this case, the source and target NVDs are connected to the leaf layer of the Clos network. Similarly, in a Layer 4 Clos network, a packet requires a maximum of 9 hops to travel from one NVD to another. In this case, the source and target NVDs are connected to the leaf layer of the Clos network. Therefore, the Clos network architecture maintains a constant overall network latency, which is crucial for communication within and between data centers. Clos topologies are horizontally scalable and cost-effective. More switches (e.g., more leaf switches and spine switches) can be added to each layer. By doing so, and by increasing the number of links between switches in adjacent layers, network bandwidth / throughput capacity can be easily increased.

[0128] In certain embodiments, each resource within the CSPI is assigned a unique identifier called a Cloud Identifier (CID). This identifier is included as part of the resource's information. This identifier can be used to manage the resource, for example, via a console or API. An example syntax for a CID is as follows:

[0129] ocid1.<RESOURCE TYPE> . <realm>[REGION] [FUTURE USE]<UNIQUE ID> That is the case. During the ceremony, "ocid1" is a string that indicates the CID version.

[0130] "resource type" refers to the type of resource (e.g., instance, volume, VCN). This represents subnets, users, and groups.

[0131] "realm" represents the area where the resource resides. As an example value, "c1" is a commercial space. The domain names represent different service areas; "c2" represents the government cloud domain, and "c3" represents the federal government cloud domain. Each domain may have its own domain name.

[0132] The "region" field represents the area to which the resource belongs. If no region applies to the resource, this field can be left blank.

[0133] "Future use" indicates that something is reserved for future use. "unique id" is the unique ID part. This format is used for resources or services. This may vary depending on the type of screw.

[0134] Scaling IP addresses in overlay networks This disclosure provides a technique for scaling IP addresses in a virtualized cloud networking environment. In certain embodiments, IP addresses are scaled in a virtualized networking environment without the use of a load balancer, thereby avoiding various problems associated with using a load balancer.

[0135] As described above, the Cloud Service Provider Infrastructure (CSPI) provided by an IaaS Cloud Service Provider (CSP) can be used to host multiple overlay networks (also known as virtual cloud networks or VCNs) for multiple customers of the CSP. A CSPI includes a physical network with multiple physical network devices such as host machines, NVDs, routers, and switches. IP addresses associated with network devices within the physical network are called physical IP addresses. An overlay network (or VCN) is a logical or virtual network operating on top of the physical network. A physical network can support one or more overlay networks or VCNs. Typically, an overlay network is customer-specific and is called a customer VCN. A customer can deploy one or more compute instances and other resources within their VCN; these compute instances may be virtual machines, bare-metal instances, etc. Overlay IP addresses are IP addresses associated with compute instances and other resources (e.g., load balancers) within the overlay network, such as compute instances deployed in the customer's VCN. Two different customers may, in some cases, use the same overlay IP address in their respective VCNs.

[0136] As described above, in certain implementations, a single overlay IP address may be associated with multiple compute instances by attaching the single overlay IP address to multiple virtual network interface cards (VNICs) and associating the VNICs with multiple compute instances. Duplicate IP addresses attached to multiple compute instances are called multi-attach IP addresses. In some other implementations, Equal-Cost Multipath (ECMP) technology in the overlay network is used to scale overlay IP addresses. Both multi-attach IP address technology and ECMP-based technology are described in more detail below.

[0137] Multi-attach IP addresses A virtualized cloud network infrastructure is provided that enables IP scaling by allowing a single overlay IP address to be attached to multiple compute instances. This allows IP addresses to scale horizontally across multiple compute instances without the need for load balancers. The new infrastructure allows traffic directed to a single multi-attach overlay IP address to be distributed across multiple server compute instances to which that single overlay IP address is attached, without the need for load balancers.

[0138] Duplicate IP addresses attached to multiple compute instances are called multi-attach IP addresses. By using a multi-attach IP address associated with a compute instance to communicate with that compute instance, for example, packets can be sent to or received from that compute instance. In certain implementations, an overlay IP address can be attached to multiple compute instances by attaching the overlay IP address to multiple virtual network interface cards (VNICs) and associating the VNICs with multiple compute instances. Traffic directed to the multi-attach IP address is then distributed across multiple compute instances without the need for a load balancer.

[0139] Processing requiring multiple attached IP addresses can be performed in stateful mode or stateless mode. Figure 6A shows a distributed cloud environment that implements scaling of overlay IP addresses using multiple attached IP addresses using stateful mode processing. Figures 7A and 7B show flowcharts illustrating the processes performed to scale multiple attached IP addresses using stateful processing. Figure 6B shows a distributed cloud environment that implements scaling of overlay IP addresses using multiple attached IP addresses in stateless mode. Figure 8 is a flowchart illustrating the processes for scaling overlay IP addresses using multiple attached IP addresses using stateless mode processing, according to several embodiments.

[0140] Figure 6A shows an example of a distributed virtualized multi-tenant cloud environment 600 that may be hosted by a CSPI provided by a Cloud Service Provider (CSP) according to a specific embodiment, and which enables scaling of overlay IP addresses using multi-attach IP addresses in stateful mode processing without the use of a load balancer. The distributed environment 600 includes multiple systems that are communicably coupled via a physical network (underlying network) or a switch fabric 610. The physical network 610 may include multiple networking devices such as multiple switches and routers that enable communication using protocols such as Layer 3 communication protocols. In some implementations, the physical network 610 may be an n-layer Clos network as shown and described above in Figure 5. The value of "n" may be 1, 2, 3, etc., depending on the implementation.

[0141] The distributed environment 600 shown in Figure 6A is merely an example and is not intended to unduly limit the scope of the claimed embodiments. Many variations, alternatives, and modifications are possible. For example, in some implementations, the distributed environment 600 may have more or fewer systems or components than those shown in Figure 6A, or it may be a combination of two or more systems, or it may have systems in different configurations or arrangements. The systems, subsystems, and other components shown in Figure 6A may be implemented using hardware or a combination thereof, with software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of each system. The software may be stored on a non-temporary storage medium (e.g., on a memory device).

[0142] The various components shown in Figure 6A are associated with IP addresses, which can be physical IP addresses or overlay IP addresses, depending on the component. In the figure, "P_IP" is used to indicate a physical IP address, and "0_IP" is used to indicate an overlay IP address. These IP addresses can be IPv4 addresses (32-bit addresses), IPv6 addresses (128-bit addresses), or other types of IP addresses. In some embodiments, IP addresses may also be represented in CIDR (Classless Inter-Domain Routing) IP address format. A CIDR address consists of two numbers written in the format "X / Y", where the prefix "X" is an IPv4 address (e.g., 10.10.10.1) and the suffix "Y" (e.g., / 12) indicates which part (how many bits) of the address is allocated to the network and is considered important for network routing. For example, 192.18.0.0 / 23 indicates that 23 bits are used for the network portion.

[0143] As a use case, a CSPI customer may provide a service (e.g., a cloud service) that is accessible by clients using a specific IP address endpoint. In the example shown in Figure 6A, we assume that the service will be accessible using the overlay IP address "B". The overlay IP address "B" can be an IPv4 address, an IPv6 address, or another type of IP address. For example, IP address B may be "10.10.10.1".

[0144] As shown in Figure 6A, the customer has provided multiple compute instances 602-1 to 602-N for scaling and implementing cloud services. The compute instances in compute instances 602-1 to 602-N can be virtual machine compute instances or bare metal compute instances. The same single overlay IP address "B" is associated with or attached to each of the multiple server compute instances 602-1 to 602-N. By allowing the same single overlay IP address "B" to be attached to multiple compute instances serving services, a single IP address is scaled across multiple service compute instances.

[0145] In Figure 6A and the following description, compute instances 602-1 to 602-N are referred to as server compute instances, but this is not intended to be limiting. A compute instance can function as a server when responding to requests from a client. The same compute instance can also function as a client when sending requests to another compute instance. Computed instances 602-1 to 602-N are referred to as server compute instances for the sake of simplicity in describing the teachings described herein.

[0146] Attaching a single overlay IP address to multiple server compute instances. There are various ways to make this possible. As mentioned above, compute instances engage with the overlay network (e.g., customer VCN) via VNICs configured for and associated with the compute instance. The VNICs may be part of a subnet of the VCN. In certain embodiments, multiple VNICs may be associated with the same compute instance, and these VNICs may be in separate VCNs. In some implementations, a single overlay IP address is attached to a compute instance by associating or attaching a single overlay IP address to a VNIC and then associating that VNIC with the compute instance. For example, a customer may register multiple server compute instances with CSPI via a command line interface, API, etc., using a console provided by CSPI. At registration, a VNIC may be configured for each compute instance in the multiple server compute instances, a single overlay IP address may be associated with the VNIC, and then the VNIC may be associated with the compute instance. In some implementations, it may be possible for the customer to specify a VNIC and a single overlay IP address (e.g., address "B") to be attached to the multiple compute instances. In several other cases, CSPI can automatically identify the overlay IP address that should be used as the multi-attach IP address associated with the VNIC and compute instance.

[0147] For example, in the example shown in Figure 6A, server compute instance #1 602-1 may be associated with VNIC #1 and the overlay IP address "B" is attached to VNIC #1; server compute instance #2 602-2 may be associated with VNIC #2 and the IP address "B" is attached to VNIC #2; server compute instance #N 602-N may be associated with VNIC #N and the IP address "B" is attached to VNIC #N, and so on. In this way, the same overlay IP address is attached to multiple VNICs and multiple compute instances associated with VNICs. Customers do not need to register all server compute instances at the same time. Customers may add or remove server compute instances and associate them with VNICs attached to multi-attach IP addresses as needed and over time.

[0148] For VNICs associated with a compute instance, the functionality that implements the VNIC is performed by a Network Virtualization Device (NVD) connected to the host machine hosting the compute instance. The NVD may run software that provides VNIC functionality (e.g., software that implements network virtualization functionality). For example, if the compute instance is a virtual machine instance hosted by a host machine, the VNIC functionality for the VNIC associated with the compute instance may be performed by an NVD connected to the host machine hosting the virtual machine compute instance. For example, as shown and described above in Figure 2, for a virtual machine compute instance 268 hosted by host machine 202, the VNIC functionality for VNIC 276 associated with the compute instance is performed by NVD 210 connected to host machine 202. As another example, in Figure 2, a bare metal instance 272 hosted by host machine 206 is associated with VNIC 280, which is performed by NVD 212 connected to host machine 206. In the case of a compute instance whose VNIC functionality is performed by an NVD, the NVD is referred to as the one associated with the compute instance or as the NVD corresponding to the compute instance. For example, in Figure 6A, NVD604-1 is associated with server compute instance 602-1 and performs functions for the VNIC (VNIC#1) associated with compute instance 602-1, NVD604-2 is associated with server compute instance #2 602-2 and performs functions for the VNIC (VNIC#2) associated with compute instance #2 602-2, and so on.

[0149] In the embodiment shown in Figure 6A, NVD604-1 to NVD604-N represent NVDs that perform VNIC-related functions for VNICs configured for server compute instances 602-1 to 602-N. Each NVD associated with a compute instance has its own associated unique physical IP address that can be used to communicate with the NVD. For example, in Figure 6A, NVD-S#1 604-1 has the physical IP address "A.1" and performs VNIC-related functions for VNIC#1, which is the VNIC associated with server compute instance #1 602-1; NVD-S#2 604-2 has the physical IP address "A.2" and performs VNIC-related functions for VNIC#2, which is the VNIC associated with server compute instance #2 602-2; NVD-S#N 604-N has the physical IP address "AN" and performs VNIC-related functions for VNIC#N, which is the VNIC associated with server compute instance #N 602-N, and so on.

[0150] As mentioned above, NVD runs software that implements a network virtualization stack, which is responsible for performing various networking and virtualization-related functions, such as performing VNIC-related functions. NVD functions as a smart NIC, a top-of-rack (TOR switch), a smart TOR (a scalable TOR), and a router, switch, or network. It can be implemented in various forms, including a general-purpose computer host that runs the virtualization stack.

[0151] In certain embodiments, the VCN control plane 616 is responsible for configuring and managing the overlay network (e.g., VCN) hosted by the CSPI. The VCN control plane 616 may store configuration information for the VCN for various customers. For example, the VCN control plane 616 may store mappings and associations between IP addresses and VNICs, between VNICs and compute instances, and between VNICs and VCNs. Using this information, the VCN control plane 616 may generate forwarding tables (e.g., forwarding table 1002 shown in Figure 10), mapping tables (e.g., IP mapping table 612 shown in Figure 6A), etc., which can be used by CSPI components to enable communication between entities involved in the overlay network (e.g., VCN) hosted by the CSPI. In certain embodiments, the VCN control plane 616 also plays a role in storing mappings between multi-attach IP addresses and their VNICs and multiple compute instances, mappings between multi-attach IP addresses and the physical IP addresses of NVDs associated with multiple compute instances, and other mappings used to handle traffic to and from multi-attach IP addresses.

[0152] In the example shown in Figure 6A, client 606 may initiate a request to access a service provided by the customer by sending a communication to IP address "B". The communication may take the form of one or more packets originating from client 606 and having destination IP address "B". Client 606 may be another computing instance, such as a virtual machine computing instance or a bare metal computing instance, or some computing device. When the word "client" is used in reference to a computing instance, it is not intended to be restrictive. The same computing instance may also function as a server computing instance. In some implementations, client 606 itself may be a computing instance with multiple attached IP addresses.

[0153] The NVD associated with the client receives request packets originated by the client. For example, in Figure 6A, NVD608 ("NVD-Client" 608) NVD-client 608 is associated with client 606 and can receive packets sent by client 606. For example, NVD-client 608 may be connected to the host machine hosting client instance 606 and may be running a network virtualization stack that implements VNIC-related functions for the VNIC associated with client 606. In the example shown in Figure 6A, client 606 is associated with the overlay IP address (O_IP) "C", and NVD-client 608, which corresponds to client 606, has the physical IP address "D".

[0154] The processes performed by the various components in Figure 6A to provide IP scaling using multi-attach IP addresses in stateful mode will be described below with reference to the flowcharts shown in Figures 7A and 7B. Figures 7A and 7B show a simplified flowchart 700 illustrating the processes performed to scale overlay IP addresses using multi-attach IP addresses using stateful mode processing, according to several embodiments. The processes shown in Figures 7A and 7B may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of each system, using hardware or a combination thereof. The software may be stored on a non-temporary storage medium (e.g., on a memory device). The methods shown in Figures 7A and 7B and described below are illustrative and not intended to be limiting. Figures 7A and 7B show various processing steps performed in a particular sequence or order, but this is not intended to be limiting. In certain alternative embodiments, the processes may be performed in several different orders, or some steps may be performed in parallel.

[0155] The processing in flowchart 700 is performed using stateful mode processing, where information about the network flows being processed by NVD (e.g., TCP network flows, UDP network flows) is stored by NVD in the relevant flow tables (e.g., flow table 614 in Figure 6A). This flow information is then used by NVD to select which of the multiple compute instances associated with a multi-attach IP address is selected to receive packets. In stateful processing, when a specific compute instance attached to a multi-attach IP address is selected to process packets belonging to a particular network flow, all subsequent packets belonging to that particular network flow are forwarded to that same specific compute instance for processing consistency. The flowchart in Figure 8 shows processing performed in stateless mode. In stateless mode, flow information is not stored by NVD and therefore is not used for packet forwarding. Stateful processing typically requires more processing and memory resources than stateless processing, but can offer a richer set of features.

[0156] Referring to Figure 7A, processing begins in 702 when the client initiates an IP packet to be sent to a specific destination overlay IP address. For example, in the embodiment shown in Figure 6A, the packet in 702 may be generated when client 606 requests access to a service accessible via the overlay IP address "B". As a result, client 606 obtains an IP packet that is initiated and has the destination overlay IP address "B".

[0157] In 704, the IP packets generated in 702 are received by the NVD associated with the client. This NVD implements the functionality for the VNIC associated with the client that originates the packets. For example, in Figure 6A, the packets are received by the NVD-client 608 connected to the host machine hosting client 606. In the embodiment shown in Figure 6A, the NVD-client 608 is physically It has the IP address "D".

[0158] In 706, the NVD that receives the IP packet in 704 refers to its IP mapping table to determine how to forward the received packet. If the packet's destination IP address is a multi-attach IP address, the NVD determines in 706 that the packet's destination IP address maps to multiple physical IP addresses of multiple NVDs. For example, in Figure 6A, the NVD-client 608 may determine, based on the information in its local IP mapping table 612, that the destination IP address of the packet received from client 606 is "B" and that the overlay IP address "B" maps to multiple physical IP addresses A.1, A.2, A.3, A.4, ..., AN.

[0159] In some embodiments, information mapping overlay IP addresses to the physical IP addresses of NVDs is maintained by a VCN control plane, such as the VCN control plane 616 shown in Figure 6A. The VCN control plane is responsible for generating IP mapping table information and publishing it to various NVDs within the overlay network. For example, the IP mapping table 612 shown in Figure 6A may be generated by the VCN control plane 616 and published to NVD-clients 608 and other NVDs within the overlay network.

[0160] In 708, the NVD receiving a packet determines a specific network flow (e.g., TCP network flow, UDP network flow) corresponding to the received packet. In certain implementations, the combined content of specific fields from the received packet defines the network flow for the packet. For example, the content of five fields (sometimes called a 5-tuple) of a packet may be determined and combined to define the network flow for the packet. In certain implementations, the five fields are the source IP address field, the source port field (specifying, for example, a Transmission Control Protocol (TCP) port or a User Datagram Protocol (UDP) port), the destination IP address, the destination port field (specifying, for example, a TCP port or a UDP port), and the protocol field (e.g., TCP is indicated by protocol number 6 (0x06) and UDP is indicated by protocol number 17 (0x11)). In other implementations, content from different sets of fields in the packet (more than five fields, fewer than five fields, five separate fields, etc.) may be combined to determine the network flow for the packet.

[0161] In 710, the NVD performs a process to determine whether the specific network flow determined in 708 is a new network flow or an existing network flow. In certain implementations, the NVD does this by comparing the specific network flow determined in 708 with network flows identified in the NVD's flow table. In this case, the NVD's flow table identifies network flows that have been previously handled or processed by the NVD. If the specific network flow determined in 708 is found in the flow table, it indicates an existing network flow; otherwise, it indicates a new network flow. In 712, based on the process performed in 710, a check is performed to determine whether the specific network flow determined in 708 is a new network flow or an existing network flow, and further processing is performed based on the result of the check. If the specific network flow is determined to be a new network flow (i.e., the specific network flow is not found in the flow table), processing proceeds to 720 in Figure 7B; otherwise, if the specific network flow is determined to already exist in the flow table, processing proceeds to 724 in Figure 7B.

[0162] For example, in the embodiment shown in Figure 6A, at 708, the NVD-client 608 may examine the combination of contents of a specific field (e.g., the 5-tuple described above) for the received packet to determine a specific network flow for the received packet. The NVD 608 may then, at 710, search the flow table 614 to determine whether the specific network flow determined at 708 is a new network flow or an existing network flow. In the example in Figure 6A, the flow table 614 shows that the TCP network flows TCP-Flow#1 (represented by a 5-tuple), TCP-Flow#2 (represented by a 5-tuple), and TCP-Flow#3 (represented by a 5-tuple) are existing network flows. Therefore, if the specific network flow determined in 708 matches one of these three network flows (for example, if the 5-tuple of the received packet matches a 5-tuple in the flow table), the received packet belongs to an existing network flow, and processing proceeds to 724 in Figure 7B; otherwise, the received packet belongs to a new network flow, and processing proceeds to 720 in Figure 7B.

[0163] Referring to Figure 7B, if at 712 the NVD determines that the packet received at 704 belongs to a new network flow, then at 720 the client NVD uses a selection technique to select one of several physical IP addresses that map to the packet's destination overlay IP address. For example, in Figure 6A, for the overlay IP address "B", NVD 608 selects one of several physical IP addresses A.1, A.2, ..., AN that map to the overlay IP address "B". Various different selection techniques may be used, such as round-robin, least-used, randomization, or other techniques. In some implementations, the selection technique may first generate a hash value by hashing the content of a specific field in the received packet, and then, based on the generated hash value, determine a specific physical IP address from among several physical IP addresses mapped to the overlay IP address. For example, hashing may involve generating a value based on the content of a field and applying a modulo function with the value "N", where "N" is the number of physical IP addresses to map to the overlay IP address. The resulting hash value may point to or index a specific physical IP address from among multiple physical IP addresses, and in 720, that specific physical IP address is selected. In certain embodiments, the goal of the selection technique is to evenly distribute traffic directed to the multi-attach overlay IP address across multiple server compute instances attached to the multi-attach overlay IP address.

[0164] In 722, the NVD may add an entry to its flow table that references a specific network flow determined in 708. This added entry may also indicate that, for a particular network flow determined in 708, the physical IP address selected in 720 was selected to process packets belonging to this network flow. This entry in the flow table may then be used for subsequent packets belonging to that particular network flow, enabling subsequent packets belonging to the network flow to be sent to the same physical IP address, and therefore to the same server compute instance for processing. In this way, the infrastructure ensures that packets belonging to the network flow reach the same server compute instance and are processed by the same server compute instance. Processing then proceeds to 726 as described below.

[0165] For example, in the embodiment shown in Figure 6A, we assume that the specific network flow determined for the received packet is TCP-Flow#4. NVD-Client 60 Based on the processing in 710 and 712, 8 determines that this network flow is not found in the flow table 614, and therefore the received packet belongs to a new network flow. The NVD-client 608 then selects one physical IP address from addresses A.1, A.2, A.3, ..., AN. As an example, let's assume that physical IP address A.2 is selected. The NVD-client 608 may then add an entry to the flow table 614 that refers to TCP-Flow#4 and further indicates that physical IP address A.2 was selected to process the packet belonging to this network flow. For example, the entry "TCP-Flow#4-A.2" may be added to the flow table 614 in 722.

[0166] If, in 704, the NVD determines in 712 that the packet received by the NVD belongs to an existing network flow, then in 724, the NVD identifies an entry in the flow table for the specific network flow determined in 708, and the specific physical IP address identified by that entry is selected as the address to which the packet should be sent. This specific physical IP address is the NVD's address associated with a specific server compute instance from among multiple server compute instances attached to the same overlay IP address. In effect, by selecting a specific physical IP address for the NVD in 724, a specific server compute instance to which the packet should be sent is selected, and this server compute instance is the instance that is already processing the specific network flow to which the received packet belongs. For example, in Figure 6, the NVD-client 608 may determine that the received packet belongs to the network flow "TCP-Flow#2" identified in the flow table 614, and then the physical IP address "A.2" is selected as the address to which the received packet should be sent.

[0167] In 726, the tunneling protocol is used to communicate packets from the NVD receiving the packets in 704 to the NVD associated with the selected physical IP address in 720 or 724. IP-in-IP (or Generic Routing Encapsulation (GRE)), Virtual Extensible LAN (V xLAN-IETF RFC7348), Virtual Private Network (VPN) (for example, MPLS Layer 3 Virtual Private Network (R) Various different tunneling protocols may be used, such as FC4364), VMware's NSX, and GENEVE (Generic Network Virtualization Encapsulation). Tunneling typically involves encapsulating a received packet in one or more headers, and the information in the headers is used to communicate the packet from the sending NVD to the receiving NVD over the physical network 610. Tunneling in a virtualized network allows traffic to be communicated from one endpoint (e.g., the NVD sending the packet) to another endpoint (e.g., the NVD receiving the packet) without making any changes to the original packet, which has been received by the sending NVD and made unrestricted by the broadcast domain or physical network. The processing related to packet encapsulation is performed by the sending NVD. For example, in Figure 6A, packet encapsulation is performed by the NVD-client 608. For example, if it is determined that a received packet belongs to the network flow "TCP-Flow#2" and should be sent to physical IP address A.2, the tunneling protocol is used to communicate the packet from NVD-client 608 to NVD-S#2 604-2 associated with physical IP address A.2.

[0168] In 728, the NVD associated with the physical IP address to which the packet was sent in 726 receives the packet. For example, if the packet is sent to physical IP address A.2, NVD-S#2 604-2 receives the packet. In 730, 728 The NVD that receives the packet then forwards the packet to the server compute instance associated with the receiving NVD and whose IP address is the multi-attach overlay IP address. For example, in Figure 6A, NVD604-2 forwards the packet to server compute instance #2 602-2 associated with overlay IP address "B".

[0169] The processes shown in Figures 7A and 7B, as well as described above, can be performed for multiple packets destined for a multi-attached overlay IP address (e.g., overlay IP address "B"), where these packets originate from one or more clients. An NVD connected to the host machine hosting the client sending the packets is configured to perform a process to determine which of the multiple NVDs associated with the multiple compute instances attached to the multi-attached IP address should receive the packet. In effect, the NVD selects a specific compute instance from among the multiple compute instances attached to the multi-attached IP address to receive the packet. Different compute instances may be selected at different times for packets addressed to the multi-attached IP address. For example, a packet belonging to the first network flow may be sent to compute instance #1, a packet belonging to the second network flow may be sent to compute instance #2, a packet belonging to the third network flow may be sent to compute instance #3, and so on. In this way, packets sent to a multi-attach IP address are distributed across multiple server computing instances, all without the use of any load balancer or complex routing tables.

[0170] In certain embodiments, the functions shown in Figures 7A and 7B, performed by the NVD (e.g., NVD-client 608 and NVD604), are implemented using software executed by one or more processors of the NVD. For example, the NVD may run network stack software that implements the various functions described herein.

[0171] As described above, Figures 7A and 7B illustrate processing performed in stateful mode, in which case the flow table is maintained by an NVD that shows the existing flow, and the information in the flow table is used to determine how packets should be forwarded. Packets belonging to a particular network flow are sent to the same server compute instance and processed by the same server compute instance. However, stateful processing requires more processing and resources than stateless processing (e.g., memory to store the flow table, extra processing required to check the flow table information, etc.).

[0172] Figure 6B shows an example of a distributed virtualization multi-tenant cloud environment 650, which can be hosted by a CSPI provided by a Cloud Service Provider (CSP) according to several embodiments, and which enables scaling of overlay IP addresses using multi-attach IP addresses in stateless mode processing without the use of a load balancer. The distributed environment 650 is very similar to the distributed environment 600 shown in Figure 6A, and similar components in Figure 6B are classified using the same reference numbers as in Figure 6A. However, in Figure 6B, there are no flow tables. Since NVD does not store any flow tables, the processing is described as stateless.

[0173] Figure 8 shows a simplified flowchart 800 illustrating a process for scaling overlay IP addresses using multi-attach IP addresses with stateless mode processing according to several embodiments. The process shown in Figure 8 uses hardware, or a combination thereof, to scale one or more processing units (e.g., processors) of each system. It can be implemented in software (e.g., code, instructions, programs) executed by the processor (core). The software can be stored on a non-temporary storage medium (e.g., on a memory device). The methods shown in Figure 8 and described below are illustrative and intended to be non-limiting. Figure 8 shows various processing steps performed in a particular sequence or order, but this is not intended to be limiting. In certain alternative embodiments, the processing may be performed in several different orders, or several steps may be performed in parallel.

[0174] The processing in 802, 804, and 806 is the same as the processing performed in 702, 704, and 706, respectively, as shown in Figure 7A and described above. In 808, a hash value is generated for the received packet. Various different hashing techniques may be used in 808. In some implementations, the hash value is generated using the content of one or more fields of the received packet. For example, an initial value may be generated using the content of the source IP address field, source port field, destination IP address field, destination port field, and protocol field of the received packet, and then the modulo "N" function may be applied to this initial value to generate the final hash value, where "N" is the number of physical IP addresses mapped to the multi-attach overlay IP address. In other implementations, the hash value may be generated using the content of different sets of fields in the packet (more than 5 fields, fewer than 5 fields, 5 separate fields, etc.). Certain implementations use consistent hashing techniques. Consistent hashing technology minimizes the impact of hash interruptions when server compute instances are added to or removed from a pool of server compute instances attached to a multi-attached IP address.

[0175] In version 820, the client NVD uses the hash value generated in version 808 to select a specific physical IP address from multiple physical IP addresses mapped to the same destination overlay IP address in the packet. In certain implementations, the hash value generated in version 808 serves as an index to a specific single physical IP address from multiple physical IP addresses mapped to a multi-attach IP address.

[0176] In 826, a tunneling protocol is used to communicate the packet from the NVD receiving the packet in 804 to the NVD associated with the physical IP address selected in 820. The processing in 820 is the same as the processing in 720 in Figure 7B described above. The processing in 828 and 830 is the same as the processing in 728 and 730 in Figure 7B described above, and the packet is forwarded in 826 to the server compute instance associated with the NVD to which the packet is sent.

[0177] Stateless processing typically requires fewer memory resources than stateful processing because flow tables are not stored or used by the NVD. Similarly, stateless processing typically requires fewer processing steps than stateful processing because flow table-related operations are not performed. As a result, in certain implementations, stateless processing tends to be faster than stateful processing.

[0178] Similar to stateful processing, in stateless processing, packets belonging to a particular network flow are sent to the same NVD (i.e., the same physical IP address is selected) and therefore to the same server compute instance for processing. This is because, as part of hashing, an initial value generated based on the content of one or more fields of the received packet uniquely identifies the network flow to which the packet belongs, and when a modulo function is applied to this initial value, the resulting hash value maps or hashes to the same physical IP address.

[0179] As described above, in a scenario where several compute instances are behind a load balancer, an external client can send packets to one or more compute instances, but the compute instances behind the load balancer cannot initiate new connections or communications with the client. Therefore, with conventional load balancers, connections to compute instances behind the load balancer must be initiated externally, but instances behind the load balancer cannot initiate connections to external clients. The novel architecture described herein overcomes this limitation. As described above, a client can initiate new communications or connections with server compute instances attached to a multi-attach IP address. A server compute instance receiving a communication initiated by a client can respond to the communication initiated by the client by sending one or more response packets to the client. In addition, a server compute instance attached to a multi-attach IP address can also initiate new communications or connections with one or more clients, in which case the new communication or connection request does not respond to previous communications from the client. NVD is configured to ensure that packets are properly communicated to the intended client receiving side for communications initiated by server compute instances attached to a multi-attach IP address. The system also performs a process to enable a client receiving a new connection or communication from a server compute instance attached to a multi-attach IP address to respond to communication initiated by that server compute instance, and the client's response (e.g., a client-outgoing response packet) is appropriately forwarded to the correct specific server compute instance that initiated the client communication. Thus, a client can initiate communication with a server compute instance using a multi-attach IP address, and a server compute instance attached to a multi-attach IP address can also initiate a connection or communication with a client (by sending packets).

[0180] For example, in Figure 6A or Figure 6B, a service accessed using the multi-attach IP address "B" may wish to push data out to one or more clients. Any of the multiple server compute instances 602-1 to 602-N associated with the multi-attach overlay IP address "B" can initiate a new connection and push data out to a client using the teachings disclosed in this disclosure. A client receiving these communications initiated by a server compute instance can respond to the communications, and this response is appropriately forwarded to the specific server compute instance that initiated the communications.

[0181] Figures 9A and 9B show simplified flowcharts 900 illustrating a process for enabling communication from a server compute instance attached to a multi-attach IP address to one or more clients, according to several embodiments. The processes shown in Figures 9A and 9B may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of each system, using hardware or a combination thereof. The software may be stored on a non-temporary storage medium (e.g., on a memory device). The methods shown in Figures 9A and 9B and described below are illustrative and not intended to be limiting. Figures 9A and 9B show various processing steps performed in a particular sequence or order, but this is not intended to be limiting. In some alternative embodiments, the processes may be performed in several different orders, or some steps may be performed in parallel.

[0182] As shown in Figure 9A, in 902, the NVD associated with the server compute instance receives the IP packet sent by the server compute instance, and the packet is a client Processing begins when the destination overlay IP address corresponding to the packet is indicated. For example, in the embodiment of Figure 6A or Figure 6B, NVD-S#2 604-2 may receive a packet from server compute instance #2 602-2, and the packet may be directed to client 606 (i.e., the destination IP address of the packet is the overlay IP address "C" of client 606).

[0183] A packet contains various fields specifying the source IP address, source port (for example, specifying a Transmission Control Protocol (TCP) or User Datagram Protocol (UDP) port), destination IP address, destination port (for example, specifying a TCP or UDP port), and protocol (for example, TCP is indicated by protocol number 6 (0x06) and UDP by protocol number 17 (0x11)), as well as other fields. The destination IP address field may contain the overlay IP address of the intended receiver or destination of the packet. The destination port field may contain the port number corresponding to the destination client's port. The source IP address field may contain the overlay IP address of the server compute instance sending the packet (i.e., a multi-attach overlay IP address). The source port field identifies the port associated with the server compute instance. The protocol field may specify a number that identifies the protocol being used (e.g., TCP, UDP). Since multiple server compute instances have the same overlay IP address, this field value will be the same for packets originating from each of multiple compute instances attached to a multi-attach IP address. The source port field may be blank, or it may contain the port number associated with the server compute instance sending the packet.

[0184] In 904, the NVD receiving the packet performs processing to determine whether the packet received in 902 is a response packet or a packet to initiate a new connection or communication between the server compute instance sending the packet and the destination client. A response packet is a packet sent by the server compute instance to the destination client as part of an existing communication or connection between the destination client and the server compute instance. A response packet is a packet sent by the destination client to the server compute instance in response to a packet previously communicated by the destination client to the server compute instance.

[0185] NVD can use various techniques to determine whether a packet received in 902 is a response packet or a packet corresponding to a new server compute instance initiation communication. According to one technique, this determination can be made based on the communication protocol used to communicate the packet. For many communication protocols, the content of a packet for initiating a new connection is different from packets that are part of an existing connection or flow. For such communication protocols, NVD can determine whether a received packet is a response packet or a packet for initiating a new connection or communication by examining the packet's content. For example, in TCP, the protocol itself distinguishes between response packets and new connection packets. If the packet is a bare SYN packet, then it is a packet for establishing a new connection. If the packet is a SYN packet with an ACK inside, then it is a packet generated in an already initiated communication session and is therefore a response packet. The presence of an acknowledgment (ACK) indicates that the packet is a response packet. This indicates.

[0186] According to another technique, in certain implementations, an NVD associated with a server compute instance may maintain a flow table for packets inbound to the server compute instance. For example, in Figure 6A, for a packet received by NVD604-1 and forwarded by the NVD to server compute instance 602-1, NVD604-1 is The flow information of those packets can be stored. In 902, if a packet received from the server compute instance is the "reverse" of a flow already present in the flow table, the packet is identified as a response packet; otherwise, the packet is identified as a new connection packet.

[0187] According to another technique, in certain implementations, if a packet received at 902 has a source port value that is a known service port (e.g., port 80) and not an ephemeral source port, the packet is identified as a response packet. Alternatively, if a packet received at 902 has a source port that is an ephemeral source port, the packet is identified as a new connection packet; otherwise, it is identified as a response packet.

[0188] Based on the processing performed in 904, in 906, it is checked whether the received packet is a new connection packet or a response packet. If it is a new connection / communication packet, processing proceeds to 908; otherwise, processing proceeds to 920 in Figure 9B. If the packet is determined to be a new connection packet, special processing is performed to set the value for the packet's source port field.

[0189] Generally, in the case of packets, the source IP address and source port of a packet are combined to uniquely identify the packet's origin. In this example, where the same overlay IP address is attached to multiple server computing instances, the source IP addresses of packets originating from each of these server computing instances will have the same source IP address, i.e., the multi-attached overlay IP address. For example, in Figure 6A, packets originating from each of server computing instances 602-1 to 602-N will have the same source IP address, i.e., overlay IP address "B". Furthermore, the source ports for packets originating from various server computing instances may also conflict with each other, i.e., they may be the same, resulting in a situation where two packets sent by two different server computing instances both have the same source IP and source port values. In such a situation, uniquely identifying the packet's origin becomes extremely difficult. Therefore, without special processing, in the case of packets originating from two or more different server compute instances attached to a multi-attach overlay IP address, the source IP address and source port may not uniquely identify the sender of the packet. This is particularly problematic when the recipient of the packet (e.g., the client to which the packet is directed) wishes to receive a response from the sending server compute instance, as there is no way to identify which of the multiple server compute instances with a multi-attach IP address sent the packet, and thus the recipient must receive a response from the client. To overcome this technical problem of source port spatial collisions that occur when the same IP address exists on multiple compute instances in a virtual network, special processing is performed in 908 and 910 to ensure that the source port for a packet does not collide with the source port for a packet originating from another server compute instance attached to the same overlay IP address.The processing performed in 908 and 910 ensures that the combination of source IP and source port is unique across multiple server compute instances attached to a multi-attach IP address, thereby uniquely identifying the specific server compute instance that sent the packet.

[0190] In 908, the ephemeral port value is determined for packets that are unique and do not conflict with ephemeral port values ​​used by other server compute instances that have multi-attach overlay IP addresses. In order to perform processing in 908, several Different technologies may be used.

[0191] In certain implementations, static port sharding techniques may be used. According to this technique, the available ephemeral source port space is divided among various compute instances with multi-attach IP addresses, and each server compute instance is assigned a range of ephemeral source port values ​​that do not overlap with the ranges assigned to other server compute instances with multi-attach IP addresses. The ephemeral source port values ​​assigned to a particular server compute instance are different from and do not conflict with the ephemeral source port values ​​assigned to other compute instances with the same overlay IP address attached.

[0192] For example, if 16 bits are used to specify a source port, then the "X" port is an ephemeral port. The ephemeral port value of X can be divided among various compute instances attached to a multi-attach IP address. For example, if there is an ephemeral port value of 48K, port values ​​0K to 10K may be assigned to the first compute instance, port values ​​10,001K to 20K may be assigned to the second compute instance, port values ​​20,001K to 40K may be assigned to the third compute instance, and so on. By performing such dissimilar and non-overlapping assignments, collisions of ephemeral port values ​​are avoided for packets sent from two different compute instances attached to the same IP address. Since the ephemeral port ranges assigned to separate compute instances do not overlap, collisions are avoided.

[0193] As part of 908, for packets received from a specific server compute instance, the NVD receiving the packet identifies a range of ephemeral port values ​​assigned to that specific server compute instance, and for that packet, a specific ephemeral port value is selected from the identified range. For example, in Figure 6A or Figure 6B, NVD-S#2 604-2 may receive a new connection packet from server compute instance #2 602-2. Then, NVD-S#2 604-2 receives a new connection packet from server compute instance #2 The range of ephemeral port values ​​assigned to 602-2 can be determined, and then, in 908, a specific ephemeral port value can be selected from the identified range.

[0194] In certain implementations, the implementation of different range allocations may be left to the customer. In some other implementations, a component of the CSPI (e.g., the VCN control plane) may control the allocation and distribute it to the NVD, thereby making the range effective. For example, upon receiving a packet, the NVD may determine the range of ephemeral port values ​​allocated to the originating compute instance, and in 908, may select a port value from this range to be used as the source port value for the packet.

[0195] In some other implementations, the CSPI may provide a service ("Ephemeral Port Service") that ensures selected ephemeral ports do not conflict. This service may have access to a database storing a set of ephemeral port values ​​that can be allocated to multiple compute instances having a multi-attach IP address. The database may be maintained by the VCN control plane and may be shared among the multiple compute instances attached to the multi-attach IP address. When the NVD receives a packet from a particular server compute instance having a multi-attach IP address, the NVD may send a request to this service requesting a lease on the set of ephemeral ports for a specific period. The service may respond by using the information stored in the database to allocate one or more ports to the requester, ensuring that the allocated ports are not used by any other compute instance having a multi-attach IP address. In addition, the service may also provide information about the server compute instance. It is possible to ensure that a port already assigned to an instance is not assigned to another requesting NVD to handle packets from other compute instances with the same IP address attached. This technique enables dynamic port sharding. A shared database is one way to implement this dynamic sharding technique. In other embodiments, other techniques such as in-memory distributed data stores may be used.

[0196] In some other implementations, customers may be required to ensure source port uniqueness for packets originating from compute instances to which the same overlay IP address is attached.

[0197] In step 910, the source port field of the packet is set to the ephemeral port value identified in step 908. In certain implementations, a packet received in step 902 may already have a value in its source port field. If the port value determined in step 908 differs from an existing value, port address translation (PAT) is performed, and the existing value is replaced with the port value determined in step 908. The processes performed in steps 908 and 910 ensure that a specific compute instance can be uniquely identified from multiple compute instances by combining the source IP address and source port for all outbound traffic sent from compute instances attached to the same IP address. The process then proceeds to step 920 in Figure 9B.

[0198] In 920, the NVD, which receives the packet from the server compute instance in 902, determines the specific physical IP address to which the packet is to be sent based on the packet's destination IP address (where the destination IP address identifies the intended client receiving side of the packet). The specific physical IP address may be determined from a routing table stored by the NVD that maps the client's destination overlay IP address to a physical IP address. The specific physical IP address may be the address of the NVD associated with the intended client receiving side. For example, in Figure 6A or Figure 6B, if the intended destination of the packet is client 606, the packet's destination IP field identifies the overlay IP address "C", and in 920, the physical IP address ("D") of NVD 608 associated with client 606 is determined.

[0199] In certain implementations, if a packet is a response packet corresponding to a specific network flow, the NVD may maintain a flow table corresponding to the traffic flow handled by that NVD that is inbound to the server compute instance. For each such flow, the flow table information may identify the physical IP address of the received traffic flow. In the outbound direction from the server compute instance, the NVD may determine the network flow to which the packet belongs, identify this flow in the flow table, and then use the flow information for this network flow to determine the specific physical IP address that was the source of the packet inbound to the compute instance. This specific physical IP address is then identified as the IP address to which the outbound packet from the server compute instance should be sent. In some other implementations, the NVD may have an IP mapping table that maps overlay IP addresses to physical IP addresses and may determine a specific physical IP address from this mapping table. In yet another implementation, the NVD may determine the next-hop physical IP address using a forwarding or routing table that provides next-hop path information for the destination of the outbound packet. For example, NVD may search its routing table to identify the physical IP address of the next hop.

[0200] In 922, the tunneling protocol is used to connect the NVD that received the packet in 902. Then, the packet is communicated to the NVD associated with the physical IP address determined in 920. Various different tunneling protocols can be used, such as IP-in-IP (or General Purpose Routing Encapsulation (GRE)), Virtual Extensible LAN (VxLAN - IETF RFC7348), Virtual Private Network (VPN) (e.g., MPLS Layer 3 Virtual Private Network (RFC4364)), VMware's NSX, and Geneve (General Purpose Network Virtualization Encapsulation). Tunneling typically involves encapsulating the received packet in one or more headers, and the information in the headers is used to communicate the packet from the sending NVD to the receiving NVD over the physical network 610. Tunneling in virtualized networks is unrestricted by the broadcast domain or physical network and allows traffic to be communicated from one endpoint (e.g., the NVD sending the packet) to another endpoint (e.g., the NVD receiving the packet) without modifying the original packet received by the sending NVD. The processing related to packet encapsulation is performed by the sending NVD. For example, in Figure 6A or Figure 6B, if a packet is received by NVD-S#2 604-2 from compute instance #2 602-2, packet encapsulation is performed by NVD-S#2 604-2. If it is determined that the packet should be sent to the physical IP address "D" of NVD608, the packet is communicated from NVD604-2 to NVD-client 606 associated with the physical IP address "D" using the tunneling protocol.

[0201] In step 924, the NVD associated with the physical IP address to which the packet was sent in step 922 receives the packet. For example, if the packet is sent to physical IP address "D", the NVD-client 608 receives the packet. In step 926, the NVD that received the packet in step 924 then forwards the packet to the intended client receiving side. For example, in Figure 6A, NVD 608 forwards the packet to client 606. Since the packet received by the client has a unique source IP address and source port combination, the client can respond to the packet received from the server compute instance, and even if a particular server compute instance is attached to a multi-attach IP address, the client's response packet is properly forwarded to the correct specific server compute instance that sent the packet.

[0202] As described above, an overlay IP address can be attached to multiple compute instances. In certain implementations, this is done by attaching the same overlay IP to multiple VNICs, and the overlay IP address is consequently attached to the compute instances associated with the VNICs. Components within the overlay network are configured to distribute traffic across multiple compute instances attached to a multi-attach IP address, all without the use of any load balancers, load balancing services, or complex routing tables. As a result, the complexity and latency associated with using load balancers are avoided. The distribution of packets directed to a multi-attach IP address is transparent to customers providing multiple server compute instances to which the IP address is attached. The architecture also offers considerable flexibility in that it can perform processing in a stateful or stateless manner using various different hashing techniques, including consistent hashing techniques.

[0203] A client can initiate new communication or connection with a server compute instance attached to a multi-attach IP address by sending a packet to the multi-attach IP address. A server compute instance that receives communication initiated by a client can respond to the client by sending one or more response packets to the client. A server compute instance can respond to communications initiated by a client. In addition, a server compute instance attached to a multi-attach IP address can also initiate new communications or connections with one or more clients, in which case the new communications or connection request will not respond to previous communications from the client. A client receiving communications initiated by a server compute instance can respond to the communications, and such responses are appropriately forwarded to the specific server compute instance that initiated the communications.

[0204] From the customer's perspective, a single overlay IP address is automatically attached to multiple VNICs and the compute instances associated with those VNICs, and traffic destined for the multi-attach IP address is transparently distributed across the multiple compute instances. Furthermore, the IP address attached to multiple interfaces and compute instances scales without the use of any load balancers or leveraging any explicit services that provide load balancing functionality, and without requiring the customer to make any changes to their compute instances or applications. The costs and technical complexities associated with load balancers are avoided. The customer can host services behind the multi-attach IP address, and the IP address can be scaled across multiple compute instances in a way that makes the IP address highly available and infinitely scalable, as desired by the customer, depending on how much traffic and how many connections each server compute instance should or is expected to handle. This scalability enables customers to serve a large number of clients, allowing them to scale up or down very easily by adding or removing one or more compute instances and attaching the same multi-attach IP addresses to the compute instances, all without relying on load balancers or complex traffic distribution schemes.

[0205] Equal-cost multipathing (ECMP) in overlay networks As described above, overlay IP addresses can be scaled in a virtual network by attaching them to multiple VNICs associated with multiple compute instances, without the need for load balancers or complex routing techniques. In certain implementations, IP can also be scaled in a virtual networking environment using ECMP techniques in the overlay network.

[0206] In conventional physical networking environments, ECMP is a routing strategy that routes traffic between the same sender and receiver across multiple paths with equal cost (for example, for the same connection or session). This allows for load balancing of traffic across multiple paths, thus providing more efficient use of available bandwidth. In certain embodiments, as described herein, ECMP technology is used in virtualized networking environments (for example, in overlay networks) to scale IP addresses.

[0207] Figure 10 shows an example of a distributed virtualization multi-tenant cloud environment 1000 that can be hosted by a CSPI provided by a cloud service provider according to a specific embodiment, and that uses ECMP technology but enables IP address scaling without the use of a load balancer. The distributed environment 1000 shown in Figure 10 has several components similar to those shown in Figure 6A, and these components are referred to using the same reference numbers as in Figure 6A.

[0208] The distributed environment 1000 shown in Figure 10 is merely an example and is not intended to unduly limit the scope of the claimed embodiments. Many variations, alternatives, and modifications are possible. For example, in some implementations, the distributed environment 1000 may be as shown in Figure 10. A system may have more or fewer components than others, or it may combine two or more systems, or it may have systems with different configurations or arrangements. The systems, subsystems, and other components shown in Figure 10 may be implemented using hardware or a combination thereof, with software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of each system. The software may be stored on a non-temporary storage medium (e.g., on a memory device).

[0209] Unlike Figure 6A, the embodiment in Figure 10 uses a forwarding table 1002 instead of an IP mapping table and a flow table. The forwarding table identifies multiple active next-hop routes to NVDs associated with server compute instances attached to a multi-attach IP address. For example, as shown in Figure 10, the forwarding table 1002 contains multiple next-hop paths to NVDs 604-1 to 604-N associated with multiple server compute instances 602-1 to 602-N attached to a multi-attach IP address (e.g., multi-attach IP address "B"). Since "N" compute instances are attached to multi-attach IP address "B", the forwarding table 1002 has "N" entries, each entry identifying the flow hash value corresponding to the network flow and the physical IP address of the next-hop NVD corresponding to the server compute instance. Each entry identifies the VCN tunnel to the next-hop NVD. Since there are N compute instances with the same multi-attach IP address "B", there are N hash values ​​representing N network flows, where each hash value is associated with a specific route to a particular next-hop NVD. In this way, the forwarding table identifies a one-to-one relationship between each NVD associated with a server compute instance and the VCN tunnel to the NVD. Thus, in the forwarding table, one overlay IP address (e.g., IP address "B") is mapped to multiple next hops.

[0210] For example, the entry "B-flow hash #1 - VCN-Tunnel-To-A.1" indicates that for destination overlay IP address "B", if the hash value of a specific field in the packet to be forwarded is "flow hash #1" (representing network flow #1), then the packet's next hop is an NVD with physical IP address A.1, and a VCN tunnel to A.1 should be used to forward the packet. As another example, the entry "B-flow hash #2 VCN-Tunnel-To-A.2" indicates that for destination IP address "B", if the hash value of a specific field in the packet to be forwarded is "flow hash #2" (representing network flow #2), then the packet's next hop is an NVD with physical IP address A.2, and a VCN tunnel to A.2 should be used to forward the packet.

[0211] As described above, hash values, based on hashing specific fields of a packet, represent various network flows. These hash values ​​serve as an index to specific next hops. For packets belonging to a specific network flow, i.e., packets with the same hash value, the packets are communicated to the same NVD and therefore to the same server computing instance for processing. For example, in Figure 10, for all packets received by NVD608 hashing to the hash value "flow hash #2", the next hop is "VCN-tunnel-to-A.2". Therefore All of these packets are sent to NVD604-2, which has the physical IP address of A.2, and NVD604-2 then forwards the packets to server compute instance 602-2.

[0212] In certain implementations, multiple next-hand IP addresses associated with a multi-attach IP address Information regarding the top is provided by the customer providing multiple server computing instances. This information may be provided, for example, via a console provided by CSPI, a command-line interface, an API, etc. This information is received by the VCN control plane 616, which then generates transfer tables such as transfer table 1002 and distributes these transfer tables to the NVDs in the overlay network.

[0213] In certain implementations, information about multiple routes is retrieved using standard routing protocols such as the Border Gateway Protocol (BGP). This information can be obtained. This information can be sent to NVD using BGP.

[0214] Figure 11 shows a simplified flowchart 1100 illustrating the process of scaling IP addresses in an overlay network using ECMP technology according to a particular embodiment. The process shown in Figure 11 can be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of each system, using hardware or a combination thereof. The software can be stored on a non-temporary storage medium (e.g., on a memory device). The methods shown in Figure 11 and described below are illustrative and not intended to be limiting. Figure 11 shows various processing steps performed in a particular sequence or order, but this is not intended to be limiting. In certain alternative embodiments, the processing may be performed in several different orders, or several steps may be performed in parallel.

[0215] Some of the processing steps in Figure 11 are similar to the processing steps shown in Figures 7A, 7B, and 8. For example, the processing performed in 1102 and 1104 is similar to the processing performed in 702 and 704 in Figure 7A, respectively. In 1106, the client NVD determines, based on entries in its forwarding table, that there are multiple next-hop paths for the destination overlay IP address of the received packet. The processing in 1108 is similar to the processing in 808 in Figure 8, where the client NVD generates a hash value by hashing the contents of one or more fields of the received packet, and the generated hash value identifies a specific network flow.

[0216] In step 1110, based on the overlay destination IP address of the received packet and the hash value generated in step 1108, the client NVD determines a specific next-hop path from among multiple next-hop paths associated with the destination IP address in the forwarding table. The specific next-hop path identified in step 1110 identifies a VCN tunnel to a specific physical IP address, and the specific physical IP address is the address of a specific NVD associated with a specific server compute instance from among multiple server compute instances associated with the same overlay IP address. For example, in Figure 10, if the destination IP address of the received packet is "B" and the hash value generated in step 1108 is "Flow Hash#2", then based on the information in forwarding table 1002, the NVD will determine a specific next-hop path. The next-hop path selected by D is "VCN-Tunnel-to-A.2," which identifies A.2 as the physical IP address of the NVD to which the packet is sent.

[0217] Next, using the tunneling protocol, the NVD that receives the packet at 1104 sends the packet to the NVD associated with the specific physical IP address determined at 1110 and identified as the packet's next hop. The processing at 1126, 1128, and 1130 is the same as the processing at 726, 728, and 730 described above, as shown in Figure 7B.

[0218] As described above, ECMP technology may be used to scale IP addresses within an overlay network. Many of the features and functions described above also apply to multi-attach IP address implementations and are available in ECMP implementations. For example, a compute instance associated with a single overlay IP address can initiate new connections or communications with clients, and clients can respond to those communications. For example, the processes shown in Figures 9A and 9B and described above also apply to ECMP-based implementations.

[0219] Various different types of ECMP techniques may be used, including n-ways ECMP, ECMP with flow-based traffic distribution, and "weighted" ECMP with flow-based traffic distribution. In certain implementation forms, health checks may be added to determine the health of the next hop. Health checks may be peer-to-peer between NVDs or initiated from a centralized service. ECMP with dynamic routing for Route-Health Injection (RHI) may be provided.

[0220] Examples of cloud architectures As mentioned above, IaaS (Infrastructure as a Service) is one specific type This is cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the internet). In the IaaS model, a cloud computing provider can host infrastructure components (e.g., servers, storage, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer)). In some cases, the IaaS provider may offer various services associated with the infrastructure components (e.g., billing, monitoring, logging, load balancing, and clustering). Therefore, since these services can be policy-driven, IaaS users may be able to implement policies to drive load balancing in order to maintain application availability and performance.

[0221] In some cases, IaaS customers can access resources and services over a wide area network (WAN), such as the internet, and install the rest of their application stack using the cloud provider's services. For example, a user can log into an IaaS platform, create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create storage buckets for workloads and backups, and install enterprise software on the VMs. Customers can use the provider's services to perform a variety of functions, including balancing network traffic, troubleshooting application issues, monitoring performance, and managing disaster recovery.

[0222] In most cases, the cloud computing model requires the involvement of a cloud provider. This cloud provider may or may not be a third-party service specializing in IaaS provision (e.g., offering, renting, or selling). Alternatively, a company can deploy a private cloud and become a provider of infrastructure services.

[0223] In some cases, IaaS deployment is the process of deploying a new application or a new version of an application to a prepared application server. IaaS deployment may also include the process of preparing the server (e.g., installing libraries, daemons, etc.). IaaS deployment is often a cloud pro The vendor manages the hypervisor layer (e.g., servers, storage, network hardware, and virtualization). Therefore, the customer may be responsible for the deployment of the OS, middleware, and / or applications (e.g., self-service virtual machines that can be spun up on demand).

[0224] In some cases, IaaS provisioning may refer to acquiring the computers or virtual hosts to be used and installing the necessary libraries or services on those computers or virtual hosts. In most cases, deployment does not include provisioning, and provisioning may need to be performed first.

[0225] In some cases, IaaS provisioning presents two distinct challenges. First, there's the challenge of provisioning an initial set of infrastructure before doing anything. Second, there's the challenge of evolving existing infrastructure after everything has been provisioned (e.g., adding new services, modifying services, removing services). In some cases, these two challenges can be addressed by enabling the declarative definition of infrastructure configuration. In other words, infrastructure (e.g., what components are needed and how these components interact) can be defined by one or more configuration files. Thus, the overall topology of the infrastructure (e.g., which resources depend on which and how they work together) can be described declaratively. In some examples, once the topology is defined, workflows can be generated to create and / or manage the different components described in the configuration files.

[0226] In some examples, infrastructure can include many interconnected elements. For example, there may be one or more virtual private clouds (VPCs), also known as core networks (e.g., configurable compute resources and / or potential on-demand pools of shared compute resources). In some examples, there may be one or more inbound / outbound traffic group rules provisioned to define how inbound and / or outbound traffic on the network is configured, and one or more virtual machines (VMs). Other infrastructure elements such as load balancers and databases may also be provisioned. Infrastructure can evolve incrementally as more and more infrastructure elements are desired and / or added.

[0227] In some examples, sequential deployment techniques may be employed to enable the deployment of infrastructure code across various virtual computing environments. Furthermore, the techniques described can enable infrastructure management within these environments. In some examples, a service team may write code that is desired to be deployed to one or more different production environments, typically many different geographical locations, sometimes even worldwide. However, in some examples, the infrastructure for deploying the code must first be configured. In some examples, provisioning may be done manually, resources may be provisioned using provisioning tools, and / or the code may be deployed using deployment tools after the infrastructure has been provisioned.

[0228] Figure 12 is a block diagram 1200 showing an exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 1202 may be communicably connected to a secure host tenancy 1204 which may include a virtual cloud network (VCN) 1206 and a secure host subnet 1208. In some examples, the service operator 1202 uses one or more client computing devices. It is possible. One or more client computing devices may be handheld mobile devices (e.g., iPhone®, mobile phones, iPad®, tablets, personal digital assistants (PDAs) or wearable devices (Google® Glass® head-mounted displays)) that run software such as Microsoft Windows Mobile® and / or various mobile operating systems such as iOS®, Windows Phone®, Android®, BlackBerry 8, and PalmOS, and that have the Internet, email, Short Message Service (SMS), BlackBerry®, or other communication protocols enabled. Alternatively, the client computing device may be a general-purpose personal computer, including personal computers and / or laptop computers that run various versions of the Microsoft Windows® operating system, Apple Macintosh® operating system, and / or Linux® operating system. The client computing device may be, for example, various GNU / Linux operating systems, e.g., Google The client computing device may be a workstation computer running a variety of commercially available UNIX® or UNIX-like operating systems, including but not limited to Chrome® OS. Alternatively or additionally, the client computing device may be other electronic devices that can communicate via the VCN1206 and / or a network with internet access, such as a thin client computer, an internet-enabled gaming system (e.g., a Microsoft Xbox® game console with or without a Kinect® gesture input device), and / or a personal messaging device.

[0229] VCN1206 may include a local peering gateway (LPG) 1210 that can communicate with Secure Shell (SSH) VCN1212 via LPG1210 included in SSH VCN1212. SSH VCN1212 may include an SSH subnet 1214, and SSH VCN1212 may communicate with control plane VCN1216 via LPG1210 included in control plane VCN1216. Furthermore, SSH VCN1212 may communicate with data plane VCN1218 via LPG1210. Control plane VCN1216 and data plane VCN1218 may be included in a service tenancy 1219 that may be owned and / or operated by an IaaS provider.

[0230] The control plane VCN1216 may include a control plane DMZ (demilitarized zone) layer 1220 that functions as a perimeter network (e.g., the portion of the corporate network between the corporate intranet and the external network). DMZ-based servers have a certain level of reliability and can contain security breaches. Furthermore, the DMZ layer 1220 may include a control plane application layer 1224 that may include one or more load balancer (LB) subnets 1222 and application subnets 1226, and a control plane data layer 1228 that may include database (DB) subnets 1230 (e.g., a front-end DB subnet and / or a back-end DB subnet). The LB subnet 1222 included in the control plane DMZ layer 1220 may be communicatively connected to the application subnet 1226 included in the control plane application layer 1224 and the internet gateway 1234 which may be included in the control plane VCN 1216. The application subnet 1226 may be communicatively connected to the DB subnet 1230 included in the control plane data layer 1228, the service gateway 1236, and the network address translation (NAT) gateway 1238. The control plane VCN 1216 may include the service gateway 1236 and the NAT gateway 1238.

[0231] The control plane VCN1216 may include the data plane application subnet 1226. It may include a mirror application layer 1240. An application subnet 1226 included in the dataplane mirror application layer 1240 may include a virtual network interface controller (VNIC) 1242 on which compute instance 1244 can run. Compute instance 1244 can communicate with application subnet 1226 of the dataplane mirror application layer 1240 to application subnet 1226 that may be included in the dataplane application layer 1246.

[0232] The data plane VCN1218 may include a data plane application layer 1246, a data plane DMZ layer 1248, and a data plane data layer 1250. The data plane DMZ layer 1248 may include an LB subnet 1222 that can be communicated to the application subnet 1226 of the data plane application layer 1246 and the internet gateway 1234 of the data plane VCN1218. The application subnet 1226 may be communicated to the service gateway 1236 and the NAT gateway 1238 of the data plane VCN1218. The data plane data layer 1250 may also include a DB subnet 1230 that can be communicated to the application subnet 1226 of the data plane application layer 1246.

[0233] The Internet gateway 1234 of the control plane VCN1216 and the Internet gateway 1234 of the data plane VCN1218 may be communicatively connected to a metadata management service 1252 which may be communicatively connected to the public internet 1254. The public internet 1254 may be communicatively connected to the NAT gateway 1238 of the control plane VCN1216 and the NAT gateway 1238 of the data plane VCN1218. The service gateway 1236 of the control plane VCN1216 and the service gateway 1236 of the data plane VCN1218 may be communicatively connected to a cloud service 1256.

[0234] In some cases, a service gateway 1236 of the control plane VCN1216 or data plane VCN1218 can make application programming interface (API) calls to a cloud service 1256 without going through the public internet 1254. API calls from the service gateway 1236 to the cloud service 1256 can be one-way. The service gateway 1236 can make API calls to the cloud service 1256, and the cloud service 1256 can send request data to the service gateway 1236. However, the cloud service 1256 may not initiate an API call to the service gateway 1236.

[0235] In some examples, a secure host tenancy 1204 may be directly connected to a service tenancy 1219, which may be independent. A secure host subnet 1208 can communicate with an SSH subnet 1214 via an LPG 1210, which enables bidirectional communication with an independent system. By connecting the secure host subnet 1208 to the SSH subnet 1214, the secure host subnet 1208 may be able to access other entities within the service tenancy 1219.

[0236] The control plane VCN1216 may allow users of service tenancy 1219 to configure or provision desired resources. Desired resources provisioned in the control plane VCN1216 may be deployed or used in the data plane VCN1218. In some examples, the control plane VCN1216 can be isolated from the data plane VCN1218, and the data plane mirror application layer 1240 of the control plane VCN1216 may be accessed via VNIC 1242, which may be included in the data plane mirror application layer 1240 and the data plane application layer 1246. It can communicate with the data plane application layer 1246 of lane VCN1218.

[0237] In some examples, a system user or customer may make requests, such as create, read, update, or delete (CRUD) operations, via the public internet 1254, which can communicate the requests to the metadata management service 1252. The metadata management service 1252 can communicate the requests to the control plane VCN 1216 via the internet gateway 1234. The requests may be received by the LB subnet 1222, which is included in the control plane DMZ layer 1220. The LB subnet 1222 may determine that the request is valid and, in response to this determination, may send the request to the application subnet 1226, which is included in the control plane application layer 1224. If the request is validated and requires a call to the public internet 1254, the call to the public internet 1254 may be sent to the NAT gateway 1238, which can make calls to the public internet 1254. Memory that the request may wish to store may be stored in the DB subnet 1230.

[0238] In some cases, the data plane mirror application layer 1240 can facilitate direct communication between the control plane VCN 1216 and the data plane VCN 1218. For example, it may be desirable that changes, updates, or other appropriate modifications to the configuration be applied to the resources contained in the data plane VCN 1218. Since the control plane VCN 1216 can communicate directly with the resources contained in the data plane VCN 1218 via VNIC 1242, it can perform changes, updates, or other appropriate modifications to the configuration.

[0239] In some embodiments, the control plane VCN1216 and the data plane VCN1218 may be included in the service tenancy 1219. In this case, the system user or customer does not have to own or operate either the control plane VCN1216 or the data plane VCN1218. Instead, the IaaS provider may own or operate the control plane VCN1216 and the data plane VCN1218, and both may be included in the service tenancy 1219. This embodiment may prevent the user or customer from interacting with other users' resources or other customers' resources by enabling network isolation. This embodiment may also allow the system user or customer to store databases privately without having to rely on the public internet 1254, which may not have the desired level of threat protection for storage.

[0240] In another embodiment, the LB subnet 1222 included in the control plane VCN 1216 may be configured to receive signals from the service gateway 1236. In this embodiment, the control plane VCN 1216 and the data plane VCN 1218 may be configured to be invoked by the IaaS provider's customers without calling the public internet 1254. The IaaS provider's customers may prefer this embodiment because the database used by the customer may be stored in a service tenancy 1219 that is controlled by the IaaS provider and can be isolated from the public internet 1254.

[0241] Figure 13 is a block diagram 1300 showing another exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 1302 (e.g., service operator 1202 in Figure 12) may include a virtual cloud network (VCN) 1306 (e.g., VCN 1206 in Figure 12) and a secure host subnet 1308 (e.g., secure host subnet 1208 in Figure 12), and a secure host tenancy 13 VCN1306 may be communicatively coupled to 04 (for example, secure host tenancy 1204 in Figure 12). VCN1306 may include a local peering gateway (LPG) 1310 (for example, LPG1210 in Figure 12) which can be communicatively coupled to Secure Shell (SSH) VCN1312 (for example, SSH VCN1212 in Figure 12) via LPG1210 included in SSH VCN1312. SSH VCN1312 may include an SSH subnet 1314 (for example, SSH subnet 1214 in Figure 12), and SSH VCN1312 can be communicatively coupled to control plane VCN1316 (for example, control plane VCN1216 in Figure 12) via LPG1310 included in control plane VCN1316. The control plane VCN1316 may be contained within a service tenancy 1319 (e.g., service tenancy 1219 in Figure 12), and the data plane VCN1318 (e.g., data plane VCN1218 in Figure 12) may be contained within a customer tenancy 1321, which may be owned or operated by a user or customer of the system.

[0242] The control plane VCN1316 may include a control plane DMZ layer 1320 (e.g., control plane DMZ layer 1220 in Figure 12) which may include an LB subnet 1322 (e.g., LB subnet 1222 in Figure 12), a control plane application layer 1324 (e.g., control plane application layer 1224 in Figure 12) which may include an application subnet 1326 (e.g., application subnet 1226 in Figure 12), and a control plane data layer 1328 (e.g., control plane data layer 1228 in Figure 12) which may include a database (DB) subnet 1330 (e.g., similar to DB subnet 1230 in Figure 12). The LB subnet 1322 included in the control plane DMZ layer 1320 may be communicatively connected to the application subnet 1326 included in the control plane application layer 1324 and to the internet gateway 1334 (e.g., internet gateway 1234 in Figure 12) which may be included in the control plane VCN 1316. The application subnet 1326 may also be communicatively connected to the DB subnet 1330, the service gateway 1336 (e.g., the service gateway in Figure 12) and the network address translation (NAT) gateway 1338 (e.g., NAT gateway 1238 in Figure 12) which are included in the control plane data layer 1328. The control plane VCN 1316 may include the service gateway 1336 and the NAT gateway 1338.

[0243] The control plane VCN 1316 may include a data plane mirror application layer 1340 (e.g., the data plane mirror application layer 1240 in Figure 12) which may include an application subnet 1326. The application subnet 1326 included in the data plane mirror application layer 1340 may include a virtual network interface controller (VNIC) 1342 (e.g., VNIC 1242) which can run a compute instance 1344 (e.g., similar to compute instance 1244 in Figure 12). The compute instance 1344 can facilitate communication between the application subnet 1326 of the data plane mirror application layer 1340 and the application subnet 1326 that may be included in the data plane application layer 1346 (e.g., the data plane application layer 1246 in Figure 12) via the VNIC 1342 included in the data plane mirror application layer 1340 and the VNIC 1342 included in the data plane application layer 1346.

[0244] The Internet gateway 1334 included in the control plane VCN 1316 may be communicably connected to a metadata management service 1352 (e.g., metadata management service 1252 in Figure 12), which may be communicably connected to the public internet 1354 (e.g., public internet 1254 in Figure 12). The public internet 1354 may be communicably connected to a NAT gateway 1338 included in the control plane VCN 1316. The service gateway 1336 included in the control plane VCN 1316 may be communicably connected to a cloud service 1356 (e.g., cloud service 1256 in Figure 12).

[0245] In some examples, the data plane VCN1318 may be included in customer tenancy 1321. In this case, the IaaS provider may provide a control plane VCN1316 for each customer, and the IaaS provider may configure a unique compute instance 1344 included in service tenancy 1319 for each customer. Each compute instance 1344 may enable communication between the control plane VCN1316 included in service tenancy 1319 and the data plane VCN1318 included in customer tenancy 1321. The compute instance 1344 may enable the deployment or use of resources provisioned in the control plane VCN1316 included in service tenancy 1319 in the data plane VCN1318 included in customer tenancy 1321.

[0246] In another example, an IaaS provider's customer may have a database residing in customer tenancy 1321. In this example, the control plane VCN 1316 may include a data plane minor application layer 1340 that may include application subnet 1326. The data plane mirror application layer 1340 may reside in data plane VCN 1318, but does not have to reside in data plane VCN 1318. That is, the data plane mirror application layer 1340 can access customer tenancy 1321, but does not have to reside in data plane VCN 1318 and does not have to be owned or operated by the IaaS provider's customer. The data plane mirror application layer 1340 may be configured to make calls to data plane VCN 1318, but does not have to be configured to make calls to any entity contained in control plane VCN 1316. The customer may wish to deploy or use resources within the data plane VCN1318 provisioned to the control plane VCN1316, and the data plane mirror application layer 1340 can facilitate the customer's desired deployment or other use of resources.

[0247] In some embodiments, a customer of the IaaS provider can apply filters to the data plane VCN 1318. In this embodiment, the customer can determine what the data plane VCN 1318 can access, and the customer may restrict access from the data plane VCN 1318 to the public internet 1354. The IaaS provider may not be able to apply filters to or control access from the data plane VCN 1318 to any external network or database. Applying filters and controls to the data plane VCN 1318 included in the customer tenancy 1321 can help isolate the data plane VCN 1318 from other customers and the public internet 1354.

[0248] In some embodiments, a cloud service 1356 can be invoked by a service gateway 1336 to access services that may not reside on the public internet 1354, on the control plane VCN 1316, or on the data plane VCN 1318. The connection between the cloud service 1356 and the control plane VCN 1316 or data plane VCN 1318 does not have to be live or continuous. The cloud service 1356 may reside on another network owned or operated by the IaaS provider. The cloud service 1356 may be configured to receive calls from the service gateway 1336 and not to receive calls from the public internet 1354. Some cloud services 1356 may be isolated from other cloud services 1356, and the control plane VCN 1316 may be isolated from cloud services 1356 that may not be located in the same region as the control plane VCN 1316. For example, control plane V CN1316 may be placed in "Region 1", and the cloud service "Deployment 12" may be placed in "Region 1" and "Region 2". When a call to Deployment 12 is made by the service gateway 1336 included in the control plane VCN1316 placed in Region 1, this call may be sent to Deployment 12 within Region 1. In this example, the control plane VCN1316 or Deployment 12 in Region 1 may not be communicatively connected to, or may not communicate with, Deployment 12 in Region 2.

[0249] FIG. 14 is a block diagram 1400 showing another exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 1402 (e.g., the service operator 1202 in FIG. 12) may be communicatively connected to a secure host tenancy 1404 (e.g., the secure host tenancy 1204 in FIG. 12) that may include a virtual cloud network (VCN) 1406 (e.g., the VCN 1206 in FIG. 12) and a secure host subnet 1408 (e.g., the secure host subnet 1208 in FIG. 12). The VCN 1406 may include an LPG 1410 (e.g., the LPG 1210 in FIG. 12) that may be communicatively connected to an SSH VCN 1412 (e.g., the SSH VCN 1212 in FIG. 12) via the LPG 1410 included in the SSH VCN 1412. The SSH VCN 1412 may include an SSH subnet 1414 (e.g., the SSH subnet 1214 in FIG. 12), and the SSH VCN 1412 may be communicatively connected to a control plane VCN 1416 (e.g., the control plane VCN 1216 in FIG. 12) via the LPG 1410 included in the control plane VCN 1416 and may be communicatively connected to a data plane VCN 1418 (e.g., the data plane 1218 in FIG. 12) via the LPG 1410 included in the data plane VCN 1418. The control plane VCN 1416 and the data plane VCN 1418 may be included in a service tenancy 1419 (e.g., the service tenant 1219 in FIG. 12).

[0250] The control plane VCN 1416 may include a control plane DMZ layer 1420 (e.g., the control plane DMZ layer 1220 in FIG. 12) that may include a load balancer (LB) subnet 1422 (e.g., the LB subnet 1222 in FIG. 12), a control plane application layer 1424 (e.g., the control plane application layer 1224 in FIG. 12) that may include an application subnet 1426 (similar to the application subnet 1226 in FIG. 12), and a control plane data layer 1428 (e.g., the control plane data layer 1228 in FIG. 12) that may include a DB subnet 1430. The LB subnet 1422 included in the control plane DMZ layer 1420 may be communicatively connected to the application subnet 1426 included in the control plane application layer 1424 and an Internet gateway 1434 (e.g., the Internet gateway 1234 in FIG. 12) that may be included in the control plane VCN 1416. The application subnet 1426 may be communicatively connected to the DB subnet 1430 included in the control plane data layer 1428, a service gateway 1436 (e.g., the service gateway in FIG. 12), and a network address translation (NAT) gateway 1438 (e.g., the NAT gateway 1238 in FIG. 12). The control plane VCN 1416 may include the service gateway 1436 and the NAT gateway 1438.

[0251] The data plane VCN 1418 may include a data plane application layer 1446 (e.g., the data plane application layer 1246 in FIG. 12), a data plane DMZ layer 1448 (e.g., the data plane DMZ layer 1248 in FIG. 12), and a data plane data layer 1450 (e.g., the data plane data layer 1250 in FIG. 12). The data plane DMZ layer 1448 may include the LB subnet 1422 that may be communicatively connected to the reliable application subnet 1460 and the untrusted application subnet 1462 of the data plane application layer 1446 and the Internet gateway 1434 included in the data plane VCN 1418. The reliable application subnet 1460 is the service gateway 1436 included in the data plane VCN 1418, the The NAT gateway 1438 and the DB subnet 1430 included in the data plane data layer 1450 can be communicated to. The untrusted application subnet 1462 can be communicated to the service gateway 1436 included in the data plane VCN 1418 and the DB subnet 1430 included in the data plane data layer 1450. The data plane data layer 1450 may include the DB subnet 1430 which can be communicated to the service gateway 1436 included in the data plane VCN 1418.

[0252] An untrusted application subnet 1462 may include one or more primary VNICs 1464(1)-(N) that can be communicated to tenant virtual machines (VMs) 1466(1)-(N). Each tenant VM 1466(1)-(N) may be communicated to each application subnet 1467(1)-(N) that may be included in each container egress VCN 1468(1)-(N) that may be included in each customer tenancy 1470(1)-(N). Each secondary VNIC 1472(1)-(N) can facilitate communication between the untrusted application subnet 1462 included in dataplane VCN 1418 and the application subnets included in container egress VCN 1468(1)-(N). Each container egress VCN 1468(1)-(N) may include a NAT gateway 1438 that can be communicated to the public internet 1454 (e.g., public internet 1254 in Figure 12).

[0253] The Internet gateway 1434 included in the control plane VCN1416 and the Internet gateway 1434 included in the data plane VCN1418 can be communicated to a metadata management service 1452 (for example, the metadata management system 1252 in Figure 12), which can be communicated to the public internet 1454. The public internet 1454 can be communicated to the NAT gateway 1438 included in the control plane VCN1416 and the NAT gateway 1438 included in the data plane VCN1418. The service gateway 1436 included in the control plane VCN1416 and the service gateway 1436 included in the data plane VCN1418 can be communicated to a cloud service 1456.

[0254] In some embodiments, the data plane VCN1418 may be integrated with the customer tenancy 1470. This integration may be useful or desirable for the IaaS provider's customers in some cases, such as when they may want support when executing code. Customers may provide code that, when executed, could be destructive, communicate with other customer resources, or cause undesirable effects. Therefore, the IaaS provider may decide whether or not to execute the code that the customer has provided to the IaaS provider.

[0255] In some examples, an IaaS provider's customer may grant the IaaS provider temporary network access and request functionality to be attached to the data plane application layer 1446. The code to perform the functionality may run on VMs 1466(1)-(N), but may not be configured to run elsewhere on the data plane VCN 1418. Each VM 1466(1)-(N) may be connected to a single customer tenancy 1470. Each container 1471(1)-(N) contained within VMs 1466(1)-(N) may be configured to run the code. In this case, a double isolation may exist (for example, containers 1471(1)-(N) run the code, and containers 1471(1)-(N) may be contained in VMs 1466(1)-(N) that are in at least an untrusted application subnet 1462), which may help prevent erroneous or undesirable code from damaging the IaaS provider's network or the networks of different customers. Container 1471 (1) to (N) may be communicably connected to customer tenancy 1470 and may be configured to send or receive data from customer tenancy 1470. Containers 1471(1) to (N) do not have to be configured to send or receive data from any other entities in the data plane VCN 1418. Once the execution of the code is complete, the IaaS provider may kill or discard containers 1471(I) to (N).

[0256] In some embodiments, a trusted application subnet 1460 may execute code that may be owned or operated by the IaaS provider. In this embodiment, the trusted application subnet 1460 may be communicatively connected to the DB subnet 1430 and configured to perform CRUD operations in the DB subnet 1430. An untrusted application subnet 1462 may be communicatively connected to the DB subnet 1430, but in this embodiment, the untrusted application subnet may be configured to perform read operations within the DB subnet 1430. Containers 1471(1)~(N), which may be included in each customer's VMs 1466(1)~(N) and may execute code from the customer, do not need to be communicatively connected to the DB subnet 1430.

[0257] In other embodiments, the control plane VCN1416 and the data plane VCN1418 do not have to be directly coupled in a communicative manner. In this embodiment, direct communication between the control plane VCN1416 and the data plane VCN1418 may not exist. However, indirect communication by at least one method may exist. An LPG1410 that facilitates communication between the control plane VCN1416 and the data plane VCN1418 may be established by the IaaS provider. In another example, the control plane VCN1416 or the data plane VCN1418 can make a call to a cloud service 1456 via a service gateway 1436. For example, a call from the control plane VCN1416 to the cloud service 1456 may include a request for a service that can communicate with the data plane VCN1418.

[0258] Figure 15 is a block diagram 1500 showing another exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 1502 (e.g., service operator 1202 in Figure 12) may be communicably connected to a secure host tenancy 1504 (e.g., secure host tenancy 1204 in Figure 12) which may include a virtual cloud network (VCN) 1506 (e.g., VCN1206 in Figure 12) and a secure host subnet 1508 (e.g., secure host subnet 1208 in Figure 12). VCN 1506 may include an LPG 1510 (e.g., LPG1210 in Figure 12) which may be communicably connected to SSH VCN 1512 (e.g., SSH VCN1212 in Figure 12) via an LPG 1510 included in SSH VCN 1512. SSH VCN1512 may include SSH subnet 1514 (for example, SSH subnet 1214 in Figure 12), and SSH VCN1512 may be communicatively connected to control plane VCN1516 (for example, control plane VCN1216 in Figure 12) via LPG1510 included in control plane VCN1516, and may be communicatively connected to data plane VCN1518 (for example, data plane 1218 in Figure 12) via LPG1510 included in data plane VCN1518. Control plane VCN1516 and data plane VCN1518 may be included in service tenancy 1519 (for example, service tenancy 1219 in Figure 12).

[0259] The control plane VCN1516 may include a control plane DMZ layer 1520 (e.g., control plane DMZ layer 1220 in Figure 12) which may include LB subnet 1522 (e.g., LB subnet 1222 in Figure 12), a control plane application layer 1524 (e.g., control plane application layer 1224 in Figure 12) which may include application subnet 1526 (e.g., application subnet 1226 in Figure 12), and a DB subnet 1530 (e.g., DB subnet 143 in Figure 14). The control plane data layer 1528 (e.g., control plane data layer 1228 in Figure 12) may include 0). The LB subnet 1522 included in the control plane DMZ layer 1520 may be communicatively connected to the application subnet 1526 included in the control plane application layer 1524 and to the internet gateway 1534 (e.g., internet gateway 1234 in Figure 12) which may be included in the control plane VCN 1516. The application subnet 1526 may be communicatively connected to the DB subnet 1530 included in the control plane data layer 1528 and to the service gateway 1536 (e.g., service gateway in Figure 12) and the network address translation (NAT) gateway 1538 (e.g., NAT gateway 1238 in Figure 12). The control plane VCN 1516 may include the service gateway 1536 and the NAT gateway 1538.

[0260] The data plane VCN 1518 may include a data plane application layer 1546 (e.g., data plane application layer 1246 in Figure 12), a data plane DMZ layer 1548 (e.g., data plane DMZ layer 1248 in Figure 12), and a data plane data layer 1550 (e.g., data plane data layer 1250 in Figure 12). The data plane DMZ layer 1548 may include an LB subnet 1522 that can be communicably connected to the trusted application subnet 1560 (e.g., trusted application subnet 1460 in Figure 14) and untrusted application subnet 1562 (e.g., untrusted application subnet 1462 in Figure 14) of the data plane application layer 1546, and to the internet gateway 1534 included in the data plane VCN 1518. A trusted application subnet 1560 may be communicatively connected to a service gateway 1536 included in data plane VCN 1518, a NAT gateway 1538 included in data plane VCN 1518, and a DB subnet 1530 included in data plane data layer 1550. An untrusted application subnet 1562 may be communicatively connected to a service gateway 1536 included in data plane VCN 1518, and a DB subnet 1530 included in data plane data layer 1550. The data plane data layer 1550 may include a DB subnet 1530 that can be communicatively connected to a service gateway 1536 included in data plane VCN 1518.

[0261] An untrusted application subnet 1562 may include primary YNICs 1564(1)-(N) that can communicate with tenant virtual machines (VMs) 1566(1)-(N) residing in the untrusted application subnet 1562. Each tenant VM 1566(1)-(N) can execute code in its respective container 1567(1)-(N) and can communicate with application subnet 1526 that may be included in a dataplane application layer 1546 that may be included in a container egress VCN 1568. Each secondary VNIC 1572(1)-(N) can facilitate communication between the untrusted application subnet 1562 included in a dataplane VCN 1518 and the application subnet included in a container egress VCN 1568. The container egress VCN may include a NAT gateway 1538 that can communicate with the public internet 1554 (e.g., public internet 1254 in Figure 12).

[0262] The Internet gateway 1534 included in the control plane VCN1516 and the Internet gateway 1534 included in the data plane VCN1518 can be communicated to a metadata management service 1552 (for example, the metadata management system 1252 in Figure 12), which can be communicated to the public internet 1554. The public internet 1554 can be communicated to a NAT gateway 1538 included in both the control plane VCN1516 and the data plane VCN1518. The service gateway 1536 included in both the control plane VCN1516 and the data plane VCN1518 can be communicated to a cloud service 1556.

[0263] In some examples, the pattern shown by the architecture in block diagram 1500 of Figure 15 may be considered an exception to the pattern shown by the architecture in block diagram 1400 of Figure 14, and may be desirable for the IaaS provider's customers when the IaaS provider cannot communicate directly with the customers (e.g., in a disconnected area). Customers can access in real time each container 1567(1)~(N) contained within each customer's VM 1566(1)~(N). Containers 1567(1)~(N) may be configured to call each secondary VNIC 1572(1)~(N) contained within the application subnet 1526 of the data plane application layer 1546, which may be contained within the container egress VCN 1568. The secondary VNICs 1572(1)~(N) can send calls to a NAT gateway 1538 which may send calls to the public internet 1554. In this example, the containers 1567(1)-(N), which customers can access in real time, may be isolated from the control plane VCN1516 and from other entities included in the data plane VCN1518. Furthermore, the containers 1567(1)-(N) may be isolated from other customers' resources.

[0264] In another example, a customer can use containers 1567(1)-(N) to invoke cloud service 1556. In this example, the customer may execute code in containers 1567(1)-(N) to request a service from cloud service 1556. Containers 1567(1)-(N) can send this request to secondary VNICs 1572(1)-(N), which can send the request to a NAT gateway that can send the request to the public internet 1554. The public internet 1554 can send this request to LB subnet 1522, which is included in control plane VCN 1516, via internet gateway 1534. In response to determining that the request is valid, the LB subnet can send this request to application subnet 1526, which can send this request to cloud service 1556 via service gateway 1536.

[0265] Note that the illustrated IaaS architectures 1200, 1300, 1400, and 1500 may include components other than those shown. Furthermore, the illustrated embodiments are only examples of some cloud infrastructure systems that may incorporate embodiments of this disclosure. In some other embodiments, the IaaS system may have more or fewer components than those shown, may combine two or more components, or may have different configurations or arrangements of components.

[0266] In certain embodiments, the IaaS system described herein may include a set of applications, middleware, and database services delivered to customers in a self-service, subscription-based, flexibly scalable, reliable, highly available, and secure manner. An example of such an IaaS system is Oracle® Cloud Infrastructure (OCI), offered by the applicant.

[0267] Figure 16 shows an exemplary computer system 1600 in which various embodiments can be implemented. System 1600 can be used to implement any of the computer systems described above. As shown, computer system 1600 includes a processing unit 1604 that communicates with a number of peripheral subsystems via a bus subsystem 1602. These peripheral subsystems may include a processing acceleration unit 1606, an I / O subsystem 1608, a storage subsystem 1618, and a communication subsystem 1624. The storage subsystem 1618 includes a tangible computer-readable storage medium 1622 and system memory 1610.

[0268] The bus subsystem 1602 provides a mechanism for various components and subsystems of the computer system 1600 to communicate with each other as intended. Although the bus subsystem 1602 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. The bus subsystem 1602 may be one of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of the various bus architectures. For example, such architectures may include the Industry Standard Architecture (ISA) bus, the Microchannel Architecture (MCA) bus, the Extended ISA (EISA) bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Interconnect (PCI) bus, which can be implemented as a mezzanine bus manufactured in accordance with the IEEE P1386.1 standard.

[0269] A processing unit 1604, which can be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of computer system 1600. The processing unit 1604 may include one or more processors. These processors may include single-core or multi-core processors. In some embodiments, the processing unit 1604 may be implemented as one or more independent processing units 1632 and / or 1634 having single-core or multi-core processors included in each processing unit. In other embodiments, the processing unit 1604 may be implemented as a quad-core processing unit formed by integrating two dual-core processors on a single chip.

[0270] In various embodiments, the processing unit 1604 can execute various programs in response to program code and can maintain multiple programs or processes that are executed simultaneously. At any given time, some or all of the program code being executed can reside in the processor 1604 and / or the storage subsystem 1618. The processor 1604 can provide the various functions described above through appropriate programming. The computer system 1600 may further include a processing acceleration unit 1606 that can include a digital signal processor (DSP), a dedicated processor, and / or the like.

[0271] The I / O subsystem 1608 may include a user interface input device and a user interface output device. The user interface input device may include a keyboard, a pointing device such as a mouse or trackball, a touchpad or touchscreen integrated into a display, a scroll wheel, a click wheel, a dial, a button, a switch, a keypad, a voice input device with a voice command recognition system, a microphone, and other types of input devices. The user interface input device may also include a motion detection and / or gesture recognition device, such as a Microsoft Kinect® motion sensor. The Microsoft Kinect® motion sensor can control and interact with input devices such as a Microsoft Xbox® 360 game controller via a natural user interface that utilizes gestures and voice commands. The user interface input device may also include an eye gesture recognition device, such as a Google Glass® blink detector. The Google Glass® blink detector detects the user's eye activity (e.g., blinking when taking a picture and / or selecting a menu) and converts the eye activity into input to an input device (e.g., Google Glass®). Furthermore, the user interface input device may include a voice recognition detection device that enables interaction between the user and a voice recognition system (e.g., Siri® Navigator) via voice commands.

[0272] Furthermore, user interface input devices include a 3D mouse, joystick or pointing stick, gamepad, graphics tablet, and speaker. This includes, but is not limited to, any audio / visual device, digital camera, digital video camera, portable media player, webcam, image scanner, fingerprint scanner, barcode reader, 3D scanner, 3D printer, laser rangefinder, and eye-tracking device. Furthermore, user interface input devices may include medical image input devices such as computed tomography scanners, magnetic resonance imaging scanners, ultrasound imaging scanners, and medical ultrasound devices. Additionally, user interface input devices may include audio input devices such as MIDI keyboards and electronic musical instruments.

[0273] The user interface output device may include non-visual displays such as display subsystems, indicator lights, or audio output devices. The display subsystem may be, for example, a flat panel device using a cathode ray tube (CRT), liquid crystal display (LCD), or plasma display, a projection device, or a touchscreen. Generally, when the term “output device” is used, it is intended to include all possible types of devices and mechanisms for outputting information from the computer system 1600 to a user or another computer. For example, the user interface output device includes, but is not limited to, various display devices that visually convey text, images, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, audio output devices, and modems.

[0274] The computer system 1600 may include a storage subsystem 1618. The storage subsystem 1618 comprises software elements, which, in the illustration, are located in system memory 1610. System memory 1610 can store program instructions that can be loaded and executed by the processing unit 1604, and data generated during the execution of these programs.

[0275] Depending on the configuration and type of the computer system 1600, the system memory 1610 may be volatile memory (e.g., random access memory: RAM) and / or non-volatile memory (e.g., read-only memory: ROM, flash memory). Generally, RAM contains data and / or program modules that the processing unit 1604 can directly access, and / or data and / or program modules currently being operated and executed by the processing unit 1604. In some implementations, the system memory 1610 is static random access memory (S It may include several different types of memory, such as RAM or dynamic random access memory (DRAM). In some implementations, a basic input / output system (BIOS) is included, which contains basic routines that help transfer information between elements within the computer system 1600, such as during startup. Generally, it can be stored in ROM. For example, and not limited to, system memory 1610 can store client applications, web browsers, middle-tier applications, and relational database management systems (RDBMs). The document also shows application programs 1612, program data 1614, and operating systems 1616, which may include S). For example, operating systems 1616 include Microsoft Windows® and Apple Macintosh®. ) and / or various versions of the Linux® operating system, various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.), and / or This includes iOS, Windows® Phone, Android® OS, and BlackBerry. This may include mobile operating systems such as Lee® 16OS and Palm® OS operating systems.

[0276] Furthermore, the storage subsystem 1618 may provide a tangible, computer-readable storage medium for storing basic programming and data structures that provide the functionality of several embodiments. Software (programs, code modules, instructions) that provides the above functionality when executed by the processor may be stored in the storage subsystem 1618. These software modules or instructions may be executed by the processing unit 1604. The storage subsystem 1618 may also provide a repository for storing data used in accordance with this disclosure.

[0277] Furthermore, the storage subsystem 1600 may include a computer-readable storage medium reader 1620 that can be further connected to the computer-readable storage medium 1622. The computer-readable storage medium 1622, together with the system memory 1610, or in combination with the system memory 1610 as needed, may comprehensively represent remote storage devices, local storage devices, fixed storage devices, and / or removable storage devices, in addition to storage media for temporarily and / or more permanently storing, storing, transmitting, and retrieving computer-readable information.

[0278] Furthermore, the computer-readable storage medium 1622 containing code or a portion of code may include any suitable medium known or used in the art. Such medium includes, but is not limited to, volatile and non-volatile, removable and non-removable media implemented in any way or technique for storing and / or transmitting information, as well as storage and communication media. This may include tangible computer-readable storage media such as RAM, ROM, electronically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or other tangible computer-readable media. It may also include intangible computer-readable media such as data signals, data transmissions, or other media that can be used to transmit desired information and are accessible by the computer system 1600.

[0279] As an example, the computer-readable storage medium 1622 may include a hard disk drive that reads from or writes to a non-removable non-volatile magnetic medium, a magnetic disk drive that reads from or writes to a removable non-volatile magnetic disk, and an optical disk drive that reads from or writes to a removable non-volatile optical disk such as a CD-ROM, DVD, or Blu-ray® disc or other optical medium. The computer-readable storage medium 1622 may also include a zip® drive, a flash memory card, or a universal serial bus (universal serial) drive. This may include, but is not limited to, USB flash drives, Secure Digital (SD) cards, DVD discs, and digital videotapes. Furthermore, computer-readable storage media 1622 may include flash memory-based SSDs, enterprise flash drives, solid-state drives (SSDs) based on non-volatile memory such as solid-state ROM, and solid-state RAM. SSDs may include those based on volatile memory such as dynamic RAM and static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory-based SSDs. Disk drives and their associated computer-readable media can provide computer system 1600 with non-volatile storage devices for computer-readable instructions, data structures, program modules, and other data.

[0280] The communication subsystem 1624 communicates with other computer systems and networks. The communication subsystem 1624 provides a interface for receiving data from other systems and transmitting data from the computer system 1600 to other systems. For example, the communication subsystem 1624 may enable the computer system 1600 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 1624 includes radio frequency (RF) transceiver components for accessing wireless voice and / or data networks (using mobile phone technologies, advanced data network technologies such as 3G, 4G, or EDGE (enhanced data rates for global evolution)). i (IEEE 802.11 family standards or other mobile communication technologies or any combination thereof), Global Positioning System (GPS) receiver It may include components and / or other components. In some embodiments, the communication subsystem 1624 may provide a wired network connection (e.g., Ethernet) in addition to, or instead of, the wireless interface.

[0281] In some embodiments, the communication subsystem 1624 may receive input communications on behalf of one or more users who may use the computer system 1600, in the form of structured and / or unstructured data feeds 1626, event streams 1628, event updates 1630, etc.

[0282] As an example, the communication subsystem 1624 handles Twitter® feeds, Facebook® updates, and Rich Site Summary (RSS) feeds. It may be configured to receive data feeds 1626, such as web feeds, in real time from users of social networks and / or other communication services, and / or to receive real-time updates from one or more third-party sources.

[0283] Furthermore, the communication subsystem 1624 may be configured to receive data in the form of a continuous data stream, which may include an event stream 1628 and / or event update 1630 of real-time events, which may be continuous or have no boundaries in an essentially definite-end state. Examples of applications that generate continuous data may include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, and automotive traffic monitoring.

[0284] Furthermore, the communication subsystem 1624 may be configured to output structured and / or unstructured data feeds 1626, event streams 1628, event updates 1630, etc., to one or more databases that can communicate with one or more streaming data source computers coupled to the computer system 1600.

[0285] Computer system 1600 can be one of a variety of types, including handheld portable devices (e.g., iPhone® mobile phones, iPad® computing tablets, PDAs), wearable devices (e.g., Google Glass® head-mounted displays), PCs, workstations, mainframes, kiosks, server racks, or other data processing systems.

[0286] Because the nature of computers and networks is constantly changing, the description of computer system 1600 shown in the figure is intended only as a specific example. Many other configurations with more or fewer components than the system shown in the figure are possible. For example, customized hardware may be used. , and / or, certain elements may be implemented in hardware, firmware, software (including applets), or a combination thereof. Furthermore, connections to other computing devices, such as network input / output devices, may be employed. Those skilled in the art will understand, based on the disclosures and teachings provided herein, other means and / or methods for implementing various embodiments.

[0287] While specific embodiments have been described, various changes, modifications, alternative configurations, and equivalents are also included within the scope of this disclosure. The embodiments are not limited to operation within a specific data processing environment, but can freely operate within multiple data processing environments. Furthermore, while embodiments have been described using a set of specific processes and steps, it should be apparent to those skilled in the art that the scope of this disclosure is not limited to the described processes and steps. The various features and aspects of the embodiments described above can be used individually or in combination.

[0288] Furthermore, while embodiments have been described using specific combinations of hardware and software, it should be recognized that other combinations of hardware and software are also included within the scope of this disclosure. Embodiments may be implemented using hardware alone, software alone, or a combination thereof. The various processes described in this disclosure can run on the same processor or on different processors in any combination. Thus, where a component or module is described as being configured to perform a particular operation, that configuration can be implemented, for example, by designing electronic circuitry to perform that operation, by programming programmable electronic circuitry (such as a microprocessor) to perform that operation, or by a combination thereof. Processes can communicate using a variety of techniques, including but not limited to prior art for inter-process communication. Different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0289] Therefore, the specification and drawings should be considered illustrative rather than restrictive. However, it will be apparent that additions, reductions, deletions, and other modifications and alterations may be made without departing from the broad spirit and scope set forth by the claims. Thus, although specific embodiments of this disclosure have been described, these embodiments are not intended to be limiting. Various examples of modifications and their equivalents are included in the appended claims.

[0290] The indefinite articles "a" and "an," the definite article "the," and similar references used in the context describing embodiments of this disclosure (particularly in the context of the claims) should be interpreted as including both singular and plural unless otherwise specifically stated herein or unless the meaning is clearly otherwise indicated. The terms "comprising," "having," "including," and "containing" are non-restrictive terms unless otherwise specifically stated. That is, it should be interpreted as "including but not limited to." The term "connected" should be interpreted as including, being attached to, or being joined together, even if something is intervening, in whole or in part. In this specification, the enumeration of value ranges is intended merely as an abbreviation to refer to each individual value that falls within that range, and unless otherwise specifically stated herein, each individual value is incorporated herein as it is described separately herein. Unless otherwise specifically stated herein or unless the meaning is clearly different in content, all methods described herein can be performed in any suitable order. In this specification, the use of any and all examples or exemplary language (e.g., "such as") is intended to make embodiments clearer and, unless otherwise specifically stated, does not limit the scope of this disclosure. Terms in the specification should be interpreted as indicating any non-claimed elements essential to the implementation of this disclosure. There isn't one.

[0291] Disjunctive language, such as the phrase "at least one of X, Y, or Z," is intended to be understood in context as generally used to indicate that an item, term, etc., may be X, Y, Z, or any combination thereof (e.g., X, Y, and / or Z), unless otherwise specified. Therefore, such disjunctive language is not generally intended, and should not, imply that a particular embodiment must contain at least one X, at least one Y, or at least one Z.

[0292] Preferred embodiments of the Disclosure, including the best known modes for carrying out the Disclosure, are described herein. Modifications of these preferred embodiments will become apparent to those skilled in the art by reading the preceding description. Those skilled in the art may, as appropriate, adopt such modifications, and the Disclosure may be carried out in ways other than those specifically described herein. Accordingly, the Disclosure includes all modifications and equivalents of the subject matter described in the claims appended herein, as permitted by applicable law. Furthermore, any combination of the above elements in all feasible modifications is incorporated herein unless otherwise indicated herein.

[0293] All references cited herein, including publications, patent applications, and patents, shall be incorporated by reference to the same extent as if they were included herein, provided that each reference is individually and clearly indicated as being incorporated by reference.

[0294] While aspects of this disclosure are described in the aforementioned specification with reference to specific embodiments, those skilled in the art will recognize that this disclosure is not limited thereto. The various features and aspects of the aforementioned disclosure may be used individually or in combination. Furthermore, embodiments may be available in any number of environments and applications beyond those described herein without departing from the broader spirit and scope of this specification. Accordingly, the specification and drawings should be considered illustrative rather than restrictive.< / realm>

Claims

1. It is a method, The destination address is the first Internet Protocol (IP) address. Regarding the first packet, which is a packet, the first network virtualization device (NVD) that receives the first packet will process multiple IP addresses The method includes the step of determining that is associated with the first IP address, wherein the first IP address is associated with a plurality of compute instances, and the plurality of IP addresses include the IP addresses of a plurality of NVDs associated with the plurality of compute instances, and the method further includes The first NVD includes the step of selecting a specific IP address from the plurality of IP addresses, wherein the specific IP address belongs to a specific NVD from the plurality of NVDs, and the specific NVD is associated with a first compute instance from the plurality of compute instances, and the method further includes, A method comprising the step of communicating the first packet to the specific IP address using the first NVD.

2. The step of determining that the plurality of IP addresses are associated with the first IP address is: The method according to claim 1, further comprising the step of determining, based on IP mapping information, that the first IP address is mapped to the plurality of IP addresses using the first NVD.

3. The step of determining that the plurality of IP addresses are associated with the first IP address is: The method according to claim 1, comprising the step of determining, based on information in a forwarding table, that a plurality of next-hop routes are associated with the first IP address, wherein the plurality of next-hop routes identify the plurality of IP addresses.

4. The method according to any one of claims 1 to 3, further comprising the step of communicating the first packet from the specific NVD to the first computing instance.

5. The step of selecting a specific IP address is, The first NVD generates a hash value based on the content of one or more fields of the first packet, The method according to any one of claims 1 to 4, further comprising the step of selecting a specific IP address from the plurality of IP addresses using the hash value.

6. The method according to claim 5, wherein the step of generating the hash value includes the step of using consistent hashing technology.

7. The step of selecting a specific IP address is, The first NVD determines that the first packet belongs to the first network flow, The first NVD identifies a first entry corresponding to the first network flow from the flow information stored by the first NVD, The method according to any one of claims 1 to 4, further comprising the step of determining that the first NVD has specified in the first entry that the particular IP address is associated with the first network flow.

8. The first NVD determines that the first packet belongs to the first network flow, The first NVD determines that the flow information stored by the first NVD does not include any entries corresponding to the first network flow, The method according to any one of claims 1 to 4, further comprising the step of selecting a particular IP address from the plurality of IP addresses using a first selection technique, based on the first NVD.

9. The method according to claim 8, further comprising the step of using the first NVD to identify the first network flow and adding an entry to the flow information that associates the specific IP address with the first network flow.

10. The first IP address is an overlay IP address, The method according to any one of claims 1 to 9, wherein the address among the plurality of IP addresses is a physical IP address.

11. The above method further, With respect to the second packet received by the first NVD, the first NVD determines that the second packet has the first IP address as its destination address, The method further includes the step of selecting a second specific IP address from the plurality of IP addresses using the first NVD, wherein the second specific IP address is of the second specific NVD from the plurality of NVDs, and the second specific NVD is associated with the second computing instance from the plurality of computing instances, and the method further includes The method according to any one of claims 1 to 10, comprising the step of communicating the second packet to the second specific IP address using the first NVD.

12. The method according to claim 11, wherein the first packet is transmitted by a first client, the second packet is transmitted by a second client, and the first client is different from the second client.

13. The above method further, The method includes the step of receiving a new packet from the first compute instance by a specific NVD from the plurality of NVDs, wherein the new packet identifies a destination IP address for a destination client, and the method further includes The method according to any one of claims 1 to 12, further comprising the step of determining, by the specific NVD, whether the new packet is for a new connection or communication to be established between the first compute instance and the destination client, or whether the new packet is a response packet sent by the first compute instance in response to a packet previously received by the first compute instance from the destination client.

14. The step of determining whether the new packet is for a new connection or a response packet includes determining whether the new packet is for a new connection or communication to be established between the first compute instance and the destination client, and the method further includes The steps include identifying the ephemeral port value to be used for the new packet, The source port field of the new packet is the identified ephemeral port value Steps to set up, The method according to claim 13, further comprising the step of communicating the new packet from the specific NVD to the NVD associated with the destination client.

15. The step of identifying the ephemeral port is: The method according to claim 14, comprising the step of selecting an ephemeral port value to be used for the new packet from a range of ephemeral port values ​​assigned to the first compute instance, wherein the ephemeral port value within the range of ephemeral port values ​​assigned to the first compute instance is different from the ephemeral port values ​​assigned to other compute instances in the plurality of compute instances.

16. The step of identifying the ephemeral port value is: The steps include sending a request to the service using the aforementioned specific NVD, which requests one or more ephemeral port values, The steps include receiving a set of one or more ephemeral port values ​​from the service using the specified NVD, The method according to claim 14, wherein the step of setting the source port field of the new packet includes setting the source port field to a value from the set of one or more ephemeral port values.

17. The step of determining whether the new packet is for a new connection or a response packet includes the step of determining that the new packet is a response packet. The method according to claim 13, further comprising the step of communicating the new packet from the specific NVD to the NVD associated with the destination client.

18. The step of determining whether the new packet is for a new connection or a response packet is: If the new packet contains an acknowledgment, the step of identifying the new packet as a response packet, If the value in the source port field of the new packet is not an ephemeral port value, the new packet is identified as a response packet, or If the particular NVD determines that the new packet is part of an existing network flow, the step of identifying the new packet as a response packet: The method according to claim 13, comprising the step of performing at least one of the steps.

19. A non-temporary computer-readable memory that stores a plurality of instructions that can be executed by one or more processors, wherein the plurality of instructions, when executed by the one or more processors, include instructions that perform the following processing, and the following processing is: The process includes determining that a first packet whose destination address is a first Internet Protocol (IP) address is associated with a plurality of IP addresses, wherein the first IP address is associated with a plurality of computing instances, and the plurality of IP addresses include the IP addresses of a plurality of network virtualization devices (NVDs) associated with the plurality of computing instances, and the process further includes: The process includes selecting a specific IP address from the aforementioned plurality of IP addresses, wherein the specific IP address belongs to a specific NVD from the plurality of NVDs, and the specific NVD is associated with a first compute instance from the plurality of compute instances, and the following process further includes: Non-temporary computer-readable memory, including the process of communicating the first packet to the specific IP address.

20. It is a system, One or more processors, The system includes a memory for storing multiple instructions, and when the multiple instructions are executed by one or more processors, the system For a first packet whose destination address is a first Internet Protocol (IP) address, the system determines that multiple IP addresses are associated with the first IP address, the first IP address is associated with multiple computing instances, and the multiple IP addresses include the IP addresses of multiple network virtualization devices (NVDs) associated with the multiple computing instances, and the multiple instructions further cause the system to: The system is instructed to select a specific IP address from the aforementioned plurality of IP addresses, the specific IP address being that of a specific NVD from the plurality of NVDs, the specific NVD being associated with a first computation instance from the plurality of computation instances, and the plurality of instructions further to the system, A system that causes the first packet to be communicated to the specific IP address.

21. Non-temporary computer-readable memory for storing a plurality of instructions executable by one or more processors, wherein the plurality of instructions, when executed by the one or more processors, include instructions that cause the one or more processors to perform the method according to any one of claims 1 to 18.

22. It is a system, One or more processors, A system comprising a memory for storing a plurality of instructions, wherein the plurality of instructions, when executed by one or more processors, cause the system to perform the method according to any one of claims 1 to 18.

Images