DNS recursive PTR signal analysis

Abstract

Network security techniques are described for monitoring a cloud environment and identifying potential issues of the monitored cloud environment, including malicious threats, using operational telemetry techniques. Techniques are described for monitoring and collecting data related to reverse or recursive DNS (rDNS) traffic associated with a monitored cloud environment. The recursive DNS traffic includes recursive DNS (rDNS) requests originating from the cloud environment and responses to the requests received from the DNS resolver. These collected data are then analyzed to identify potential threats to the monitored cloud environment. The collected data may be analyzed to identify potential threat sources and identify one or more portions of the cloud environment that are targeted for threats. The analysis may trigger generation of alarms, execution of actions (e.g., protective measures), generation of reports, discrimination of patterns, and the like.

Description

[0001] Cross-reference to related applications

[0002] This application is a PCT application that claims priority to U.S. Patent Application No. 18 / 367,938, filed September 13, 2023, the contents of which are hereby incorporated herein by reference in their entirety for all purposes. Technical Field

[0003] This disclosure generally relates to cybersecurity techniques, and more specifically to DNS recursive PTR signal analysis. Background Technology

[0004] The adoption of cloud services has grown rapidly over the past few years. This has led to an increasing number of cloud service providers (CSPs) offering one or more cloud services to their subscribers. In a typical scenario, a CSP provides a cloud environment that includes the infrastructure it provides to deliver one or more services to its customers. A cloud environment can include networked computing resources, storage resources, networking resources, software resources, and other types of resources used to provision cloud services. A cloud environment typically includes a physical network layer (called the base layer) on top of which one or more virtual networks are supported and used to provide cloud services.

[0005] Due to their distributed nature and complexity, cloud environments provided by CSPs are highly vulnerable to malicious cyberattacks. Protecting their cloud environments from cyberattacks launched by bad actors is crucial for CSPs. This is important for protecting customer data and other customer resources entrusted to them. Negative publicity resulting from security vulnerabilities can destroy a CSP's business. Therefore, CSPs are constantly seeking new and innovative ways to better protect their cloud environments. Summary of the Invention

[0006] This disclosure relates to cybersecurity technologies, and more particularly, to techniques for monitoring cloud environments and using operational telemetry to identify potential problems (including malicious threats) in the monitored cloud environment. In some embodiments, techniques are described for monitoring and collecting data related to reverse or recursive DNS (rDNS) traffic associated with the monitored cloud environment. Recursive DNS traffic includes recursive DNS (rDNS) requests originating from the cloud environment and responses to these requests received from a DNS resolver. This collected data is then analyzed to identify potential threats to the monitored cloud environment. The collected data can be analyzed to identify potential threat sources and to identify one or more portions of the cloud environment that are targets of threats.

[0007] This disclosure relates to monitoring DNS recursive resolver traffic, specifically PTR record resolution. Such monitoring allows tracking of the zones, VCNs, and / or host machines associated with each rDNS request to determine how many zones, VCNs, and / or host machines are attempting to resolve each PTR record. After data collection, observations can be aggregated, analyzed, and / or stored for later use. Analysis of rDNS request and response data can include identifying systems targeted by anomalous activity (e.g., malicious actors, abnormal activity) and / or determining what and / or who is causing the anomalous activity. As a result of the analysis, alerts can be generated, actions (e.g., protective measures) can be performed, reports can be generated, patterns can be identified, etc. Various embodiments are described herein, including methods, systems, non-transitory computer-readable storage media storing programs, code, or instructions executable by one or more processors. Some embodiments can be implemented using a computer program product comprising computer programs / instructions that, when executed by a processor, cause the processor to perform any of the methods described in this disclosure.

[0008] In some implementations, the technology (e.g., methods, systems, computer-readable media) includes: monitoring reverse DNS traffic associated with a monitored environment by a cloud defense system, the reverse DNS traffic including a set of one or more reverse DNS resolver requests originating from the monitored environment and a set of one or more responses generated by one or more DNS resolvers in response to the set of one or more reverse DNS resolver requests. The technology may also include monitoring one or more responses to the set of one or more reverse DNS resolver requests by the cloud defense system. The technology may further include: collecting and storing raw data based on the monitoring of reverse DNS traffic by the cloud defense system; enhancing the raw data by the cloud defense system to generate enhanced data; and using the enhanced data by the cloud defense system to identify anomalous network activity associated with the monitored environment. The technology may also include outputting a signal indicating anomalous network activity by the cloud defense system.

[0009] In some implementations, identifying anomalous network activity includes identifying a portion of a monitored environment experiencing anomalous network activity, the portion of the monitored environment comprising one or more components of the monitored environment. In some implementations, the one or more components of the monitored environment include at least one of the following: a Virtual Cloud Network (VCN) within the monitored environment, a region within the monitored environment, a group of one or more VCNs associated with a customer of a cloud service provider, a data center within the monitored environment, virtual machines, and host machines.

[0010] In some implementations, identifying anomalous network activity includes identifying the source of the anomalous network activity. In some implementations, the source is part of the monitored environment. In some implementations, the source is a component outside the monitored environment.

[0011] In some implementations, identifying a source includes performing the identification of at least one of the following: a first IP address associated with a source that triggered at least one of the group of one or more reverse DNS resolver requests, a first fully qualified domain name (FQDN) associated with the first IP address, or an owner associated with the first FQDN.

[0012] In some implementations, the technology also includes a set of one or more actions initiated by the cloud defense system in response to an output signal indicating anomalous network activity. In some implementations, the set of one or more actions performs at least one of the following: (i) changing a set of rules associated with a component of the cloud defense system, (ii) isolating a system within the cloud service provider infrastructure (CSPI), and (iii) causing an alert to be generated. In some implementations, the alert is a report, and the alert is sent to a user of the cloud defense system or a customer of the CSPI.

[0013] In some implementations, identifying anomalous network activity includes: generating a first baseline using prior enhancement data, which was generated before the first baseline was generated; determining a deviation from the first baseline; and identifying the deviation as anomalous network activity. In some implementations, the first baseline identifies a portion of the monitored environment and a first threshold associated with that portion of the monitored environment, and determining a deviation includes determining, based on the enhancement data, that the first threshold associated with that portion has been exceeded. In some implementations, the first baseline represents the number of rDNS requests transmitted by that portion of the monitored environment within a group of one or more rDNS resolver requests. In some implementations, the first baseline represents the number of rDNS requests transmitted by that portion of the monitored environment within a group of one or more rDNS resolver requests for resolving a group of one or more IP addresses. In some implementations, the first baseline differs from a second baseline, which identifies a second portion of the monitored environment and a second threshold different from the first threshold.

[0014] In some implementations, the group of one or more reverse DNS resolver requests is generated by one or more VCNs, one or more zones, or one or more virtual machines.

[0015] In some implementations, when generating augmented data, both the original data and an external data source are used.

[0016] The foregoing and other features and embodiments will become clearer when referenced to the following description, claims and drawings. Attached Figure Description

[0017] Figure 1 This is a high-level diagram illustrating a distributed environment of a virtual or overlay cloud network hosted by a cloud service provider infrastructure, according to certain embodiments.

[0018] Figure 2 A simplified architecture diagram of the physical components in the physical network within the CSPI according to certain embodiments is depicted.

[0019] Figure 3 An example arrangement within CSPI according to certain embodiments is shown, in which a host machine is connected to multiple network virtualization devices (NVDs).

[0020] Figure 4 The invention describes, according to certain embodiments, connectivity between multi-tenant host machines and NVDs for providing I / O virtualization to support multi-tenant host machines.

[0021] Figure 5 A simplified block diagram of a physical network provided by CSPI according to certain embodiments is depicted.

[0022] Figure 6 A simplified flowchart of a system and processes involving rDNS requests and rDNS responses, according to some embodiments, is depicted.

[0023] Figure 7 A simplified flowchart illustrating a system and process for obtaining monitored traffic data from rDNS requests and rDNS responses, according to some embodiments, is provided.

[0024] Figure 8 A simplified architecture diagram of a system that communicates with and constitutes a cloud defense system according to some embodiments is depicted.

[0025] Figure 9 A simplified flowchart 900 depicts a method for protecting a monitored environment using rDNS traffic, according to some embodiments.

[0026] Figure 10 A simplified flowchart is depicted according to some embodiments for monitoring a monitored environment to determine baseline network activity and determining actions to be taken if anomalous activity is identified.

[0027] Figure 11 A simplified flowchart for determining the baseline rDNS request volume for a monitored environment, according to some embodiments, is depicted.

[0028] Figure 12A simplified flowchart for detecting anomalous network behavior in a monitored environment, according to some embodiments, is depicted.

[0029] Figures 13A-13B A simplified flowchart for detecting malicious actors using a cloud defense system, according to some embodiments, is depicted.

[0030] Figure 14 This is a block diagram illustrating a pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0031] Figure 15 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0032] Figure 16 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0033] Figure 17 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0034] Figure 18 This is a block diagram illustrating an example computer system according to at least one embodiment. Detailed Implementation

[0035] In the following description, specific details are set forth for purposes of explanation in order to provide a thorough understanding of certain embodiments. However, it will be clear that various embodiments may be practiced without these specific details. The accompanying drawings and description are not intended to be limiting. The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment or design described herein as “exemplary” is not necessarily to be construed as preferred over or superior to other embodiments or designs.

[0036] This disclosure relates to cybersecurity technologies, and more particularly, to techniques for monitoring cloud environments and using operational telemetry to identify potential problems (including malicious threats) in the monitored cloud environment. In some embodiments, techniques are described for monitoring and collecting data related to reverse or recursive DNS (rDNS) traffic associated with the monitored cloud environment. Recursive DNS traffic includes recursive DNS (rDNS) requests originating from the cloud environment and responses to these requests received from a DNS resolver. This collected data is then analyzed to identify potential threats to the monitored cloud environment. The collected data can be analyzed to identify potential threat sources and to identify one or more portions of the cloud environment that are targets of threats.

[0037] In some implementations, a cloud defense system is provided, configured to monitor and collect data related to rDNS requests originating from a cloud environment and corresponding responses generated by a DNS resolver. The cloud defense system is configured to augment the collected raw data with additional data obtained from one or more additional data sources to generate augmented data. The cloud defense system then uses the augmented data to identify portions of the monitored cloud environment that behave anomalously and are therefore potentially targets of malicious attacks. In some implementations, the cloud defense system uses the augmented data to generate a baseline for the monitored cloud environment over a time period. In some implementations, multiple different baselines may be generated for different portions of the monitored cloud environment. The cloud defense system then identifies deviations from the baselines and identifies those portions of the monitored cloud environment experiencing deviations as potential targets of malicious attacks.

[0038] In some implementations, the cloud defense system also uses enhanced data to identify threat sources to the monitored cloud environment. For example, certain IP addresses causing abnormally high rDNS traffic can be identified as potential malicious actors. The threat source may be located outside or inside the monitored cloud environment. In some implementations, multiple distinct baselines can be generated for sources sending traffic to the monitored cloud environment. The cloud defense system then identifies deviations from such baselines and can identify those sources associated with these deviations as potential sources of malicious attacks.

[0039] Cloud defense systems reduce false positives for threats. For example, various normal activities may lead to rDNS requests. Examples include scans performed by a source. In some cases, scans may be performed by a legitimate source (e.g., a scan performed by a trusted system (e.g., an internal cybersecurity team) to identify system vulnerabilities), while in others, scans may be performed by malware seeking to spread to multiple host machines or servers and / or virtual machines. By using baselines generated over a period of time and using these baselines to find deviations, the cloud defense system described in this paper is able to identify activities constituting a genuine threat from among other normal activities. In this way, cloud defense systems reduce the occurrence of false positives.

[0040] The cloud defense system collects rDNS traffic-related data from the monitored environment and further enhances the collected data using additional data sources. This enhanced data provides a contextual view of the state of the monitored environment, which helps to more accurately identify potentially attacked portions of the monitored environment and / or sources of potential malicious attacks on at least a portion of the monitored environment. Based on this context, the cloud defense system is able to identify the intent behind data communications with the monitored environment and distinguish malicious activity from normal baseline activity. This improves the overall efficiency and availability of the cloud defense system. In addition to improving efficiency by reducing false positives, efficiency can also be improved by removing noise from network traffic data before processing (filtering network traffic data), thereby reducing the time and resources required to process filtered network traffic data compared to unfiltered network traffic data.

[0041] When abnormal activity is identified, whether a portion of the monitored cloud environment is identified as experiencing abnormal behavior, or when a specific source is identified as the source of the abnormal behavior experienced by a portion of the monitored cloud environment, the cloud defense system can initiate one or more actions. For example, one or more actions can be initiated to isolate the portion of the cloud environment experiencing abnormal behavior, contain the abnormal activity within a portion of the cloud environment, or correct network intrusions or threats. These actions may include, for example, setting up a firewall, taking a host machine offline, setting a list of IP addresses or FQDNs to be blocked, or isolating a specific VCN or a group of VCNs.

[0042] The cloud defense system is configured to collect and store rDNS traffic-related data at different levels of the hierarchical structure of the monitored cloud environment. For example, in some embodiments, the cloud defense system collects rDNS traffic data at each Virtual Cloud Network (VCN) level, where a VCN may include multiple host machines executed by one or more host machines. The cloud defense system monitors rDNS requests originating from VCNs within the monitored cloud environment and the corresponding DNS resolver responses. Data collected at the VCN level can be used to identify anomalous behavior at the VCN level. For VCNs within a data center, the collected VCN data can be aggregated to represent data center data. Data center-level aggregated data can be used to identify anomalous network activity at the data center level. For one or more data centers in a region, the collected data center data can be aggregated to represent region data. Region-level aggregated data can be used to identify anomalous network activity at the region level. For a global region comprising one or more regions, the collected region data can be aggregated to represent global region data. Global region-level aggregated data can be used to identify anomalous network activity at the global region level. In this way, the cloud defense system is able to collect, aggregate, and analyze rDNS traffic data collected across different hierarchical architecture levels of the monitored cloud environment. The collected data can also be used to monitor anomalous activity at the customer level, where a customer can be associated with one or more VCNs.

[0043] In some embodiments, the cloud defense system may include multiple systems, including a traffic monitoring system, a data enhancement system, a data analysis system, a report generator system, and a query system. The traffic monitoring system can monitor and collect data related to rDNS traffic associated with the monitored cloud environment. The monitored rDNS traffic may include rDNS requests originating from the monitored cloud environment and corresponding responses received from one or more DNS resolvers. Monitoring and collection can be performed at different tiers within the cloud environment, including targeting VCNs, customers, data centers, regions, global regions, etc.

[0044] In some implementations, the data augmentation system is responsible for augmenting the raw data collected by the traffic monitoring system to generate augmented data. Various methods exist for augmenting raw data to generate augmented data. In some use cases, the data augmentation system may use data obtained from one or more data sources, including external third-party data sources, to augment the raw data.

[0045] In some implementations, the data analytics system is responsible for analyzing augmented data and outputting analysis results. Data can be analyzed along different dimensions and parameters. For example, the data analytics system can analyze augmented data to determine baseline information for a monitored cloud environment. For instance, the baseline can identify how many rDNS requests originate from the monitored cloud environment for a given IP address under normal operating conditions. Baseline information may include information related to one or more parameters, such as the number of rDNS requests and / or responses associated with a specific IP, a specific fully qualified domain name (FQDN), or a specific owner of one or more FQDNs. The baseline information can identify one or more thresholds associated with these different parameters. The data analytics system can then identify and track deviations from baseline behavior. When a deviation from baseline behavior exceeds a pre-established threshold, the deviation can be flagged as anomalous behavior by the data analytics system. The data analytics system can identify a portion of the monitored cloud environment experiencing anomalous behavior (e.g., a specific VCN, VCN group, specific customer, data center, region, etc.). The data analytics system can also identify sources responsible for causing or triggering the anomalous behavior (e.g., a specific IP address, FQDN, entity, etc.). In some embodiments, the data analytics system can use augmented data to identify and assess the presence of one or more patterns in the collected data and the augmented data. Deviations from normal patterns can be identified as anomalous and potentially malicious behavior.

[0046] In some embodiments, after detecting anomalous behavior, the data analytics system can send a signal to a downstream system, which then takes action based on the signal received from the data analytics system. For example, the data analytics system can send a signal indicating anomalous behavior to a downstream alerting system, which in turn can generate one or more alerts in response. Based on the information contained in the signal received from the data analytics system, the alerts can be associated with different severity levels. In some embodiments, the alerting system can be configured with rules and / or machine learning models that the system uses to determine what alerts to generate, when to generate them, the content of the alerts, the recipients of the alerts, and the communication channels used to deliver the alerts to their intended recipients.

[0047] As another example, a data analytics system can send signals indicating anomalous behavior to a downstream action system, which in turn can initiate one or more actions. These actions can include actions to mitigate or isolate the target or source of the anomalous behavior, preventative actions, corrective actions, etc. In some implementations, the action system can be configured with rules and / or machine learning models that the system uses to determine what actions to initiate, when to initiate them, and the target of the actions (e.g., which VCN, host machine, etc.). Examples of actions include isolating a VCN to prevent it from interacting with other systems in the monitored cloud environment, disconnecting a specific host machine from the network, setting up firewalls, and other actions.

[0048] As another example, a data analysis system can send signals indicating anomalous behavior to a report generation system. In response, the report generation system can generate one or more reports and send them to pre-configured recipients. In some implementations, the report generator system can be configured with rules and / or machine learning models that the system uses to determine what reports to generate, when to generate them, the content of the reports, the recipients of the reports, and the communication channels used to deliver the reports to their intended recipients.

[0049] In some embodiments, the cloud defense system may provide a query system that enables users of the cloud defense system to query raw and / or augmented data and run their own data analytics. The query system may support different types of queries, such as SQL queries, natural language queries, etc. Examples of queries include those targeting: identifying all instances of reverse DNS requests generated for a specific IP address within a specific time period; the number of rDNS requests originating from a specific VCN within a specific time period; all FQDNs involved in rDNS requests exceeding a specific threshold within a specific time period; and so on. The query system enables users of the cloud defense system to obtain relevant information from the collected and augmented data, run their own analytics, etc.

[0050] The embodiments described herein are able to classify network activity in a monitored cloud environment by analyzing rDNS requests and responses, and perform processing that allows for a better understanding of the root causes of incidents and the intent behind such incidents.

[0051] This disclosure describes a novel solution for protecting cloud environments from malicious attacks, which uses techniques to monitor, collect, and analyze data related to rDNS traffic associated with the cloud environment. Because the data is collected at the VCN level, the identity of the host machine running on the VCN (which may be associated with a customer of the cloud service) is not available to the cloud defense system. This is important for many cloud service customers who want to keep their payload information confidential.

[0052] The various solutions described in this disclosure provide novel methods for protecting cloud environments using data collected from monitoring rDNS requests and / or responses. Certain embodiments can reduce processing time for analyzing network activity and improve the accuracy of detecting anomalous network activity (e.g., malicious or abnormal network activity). By identifying anomalous network activity, the uptime of the network and its systems can be increased, and the protection of its data from threats can be enhanced. In some embodiments, certain network traffic can be blocked due to actions taken by the cloud defense system, thereby allowing for enhanced connectivity to systems connected to the cloud service provider's infrastructure.

[0053] Furthermore, anomalous activity identified using the scale of cloud defense systems can be relayed to customers or users of cloud infrastructure, allowing them to obtain similar benefits.

[0054] Example Virtual Networking Architecture

[0055] The term cloud service generally refers to services provided by a cloud service provider (CSP) that make available to users or customers on demand (e.g., via a subscription model) using systems and infrastructure (cloud infrastructure) provided by the CSP. Typically, the servers and systems that make up the CSP's infrastructure are separate from the customer's own on-premises servers and systems. Therefore, customers can utilize cloud services provided by the CSP without having to purchase separate hardware and software resources for the service. Cloud services are designed to provide subscribers with simple, scalable access to applications and computing resources without requiring customers to invest in the infrastructure used to provide the service.

[0056] There are several cloud service providers that offer various types of cloud services. There are various different types or models of cloud services, including Software as a Service (SaaS), Infrastructure as a Service (IaaS), Platform as a Service (PaaS), etc.

[0057] A customer can subscribe to one or more cloud services provided by a CSP. A customer can be any entity, such as an individual, organization, or enterprise. When a customer subscribes to or registers for a service provided by a CSP, a lease or account is created for that customer. The customer can then access one or more subscribed cloud resources associated with that account.

[0058] As mentioned above, Infrastructure as a Service (IaaS) is a specific type of cloud computing service. In the IaaS model, a CSP provides infrastructure (called Cloud Service Provider Infrastructure or CSPI) that customers can use to build their own customizable networks and deploy customer resources. Therefore, the customer's resources and network are hosted in a distributed environment by the infrastructure provided by the CSP. This differs from traditional computing, where the customer's resources and network are hosted by the infrastructure provided by the customer.

[0059] CSPI can include interconnected high-performance computing resources, including various host machines, memory resources, and network resources forming a physical network, also known as the base network or underlying network. Resources in CSPI can be distributed across one or more data centers, which may be geographically dispersed across one or more geographic regions. Virtualization software can be executed by these physical resources to provide a virtualized distributed environment. Virtualization creates overlay networks (also known as software-based networks, software-defined networks, or virtual networks) on the physical network. The CSPI physical network provides the underlying foundation for creating one or more overlay or virtual networks on top of the physical network. The physical network (or base network or underlying network) includes physical network devices such as physical switches, routers, computers, and host machines. An overlay network is a logical (or virtual) network that runs on top of the physical base network. A given physical network can support one or more overlay networks. Overlay networks typically use encapsulation techniques to distinguish traffic belonging to different overlay networks. Virtual or overlay networks are also known as Virtual Cloud Networks (VCNs). Virtual networks use software virtualization technologies (e.g., hypervisors, virtualization capabilities implemented by network virtualization devices (NVDs) (e.g., smartNICs), top-of-rack (TOR) switches, intelligent TORs that implement one or more functions performed by NVDs, and other mechanisms) to create a layer of network abstraction that can run on top of the physical network. Virtual networks can take many forms, including peer-to-peer networks, IP networks, etc. Virtual networks are typically either Layer-3 IP networks or Layer-2 VLANs. This method of virtual or overlay networking is often referred to as virtual or overlay Layer-3 networking. Examples of protocols developed for virtual networks include IP-in-IP (or Generic Routing Encapsulation (GRE)), Virtual Extensible LAN (VXLAN—IETF RFC 7348), Virtual Private Networks (VPNs) (e.g., MPLS Layer-3 Virtual Private Network (RFC 4364)), VMware's NSX, GENEVE (Generic Network Virtualization Encapsulation), etc.

[0060] For IaaS, the infrastructure provided by a CSP (Center for Service Providers) can be configured to offer virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, cloud service providers can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, IaaS providers can also provision a wide variety of services to accompany those infrastructure components (e.g., billing, monitoring, logging, security, load balancing, and clustering). Therefore, since these services can be policy-driven, IaaS users can implement policies to drive load balancing to maintain application availability and performance. CSPI provides infrastructure and a complementary set of cloud services, enabling customers to build and run a wide range of applications and services in a highly available, managed, distributed environment. CSPI provides high-performance computing resources and capabilities, as well as storage capacity, in a flexible virtual network securely accessible from various networked locations, such as from the customer's on-premises deployment network. When a customer subscribes to or registers for IaaS services provided by CSP, the lease created for that customer is a secure and isolated partition within CSP, in which the customer can create, organize, and manage their cloud resources.

[0061] Customers can build their own virtual networks using the compute, storage, and networking resources provided by CSPI. One or more customer resources or workloads, such as compute instances, can be deployed on these virtual networks. For example, customers can use resources provided by CSPI to build one or more customizable and private virtual networks, known as Virtual Cloud Networks (VCNs). Customers can deploy one or more customer resources, such as compute instances, on their VCNs. Compute instances can take the form of virtual machines, bare metal instances, etc. Therefore, CSPI provides the infrastructure and a complementary set of cloud services that enable customers to build and run a wide range of applications and services in a highly available virtual hosting environment. Customers do not manage or control the underlying physical resources provided by CSPI, but they do have control over the operating system, storage devices, and deployed applications; and possibly limited control over selected networking components, such as firewalls.

[0062] The CSP can provide a console that enables customers and network administrators to configure, access, and manage resources deployed in the cloud using CSPI resources. In some embodiments, the console provides a web-based user interface for accessing and managing the CSPI. In some implementations, the console is a web-based application provided by the CSP.

[0063] CSPI can support single-tenant or multi-tenant architectures. In a single-tenant architecture, software (e.g., applications, databases) or hardware components (e.g., host machines or servers) serve a single customer or tenant. In a multi-tenant architecture, the software or hardware components serve multiple customers or tenants. Therefore, in a multi-tenant architecture, CSPI resources are shared among multiple customers or tenants. In a multi-tenant scenario, precautions and safeguards are implemented within CSPI to ensure that each tenant's data is isolated and remains invisible to other tenants.

[0064] In a physical network, a network endpoint (“endpoint”) is a computing device or system that is connected to and communicates with the physical network. Network endpoints in a physical network can be connected to a local area network (LAN), a wide area network (WAN), or other types of physical networks. Examples of traditional endpoints in a physical network include modems, hubs, bridges, switches, routers and other networking devices, physical computers (or host machines), etc. Each physical device in a physical network has a fixed network address that can be used to communicate with the device. This fixed network address can be a Layer-2 address (e.g., a MAC address), a fixed Layer-3 address (e.g., an IP address), etc. In a virtualized environment or in a virtual network, endpoints can include various virtual endpoints, such as virtual machines hosted by components of the physical network (e.g., hosted by physical host machines). These endpoints in a virtual network are addressed using overlay addresses, such as overlay addresses (e.g., overlay MAC addresses) and overlay addresses (e.g., overlay IP addresses). Network overlays enable flexibility by allowing network administrators to move the overlay addresses associated with network endpoints using software management (e.g., via software implementing a control plane for virtual networks). Therefore, unlike in a physical network, in a virtual network, overlay addresses (e.g., overlay IP addresses) can be moved from one endpoint to another using network management software. Since a virtual network is built on top of a physical network, communication between components within a virtual network involves both the virtual network and the underlying physical network. To facilitate such communication, CSPI components are configured to learn and store mappings that map overlay addresses in the virtual network to actual physical addresses in the base network, and vice versa. These mappings are then used to facilitate communication. Client traffic is encapsulated to facilitate routing within the virtual network.

[0065] Therefore, physical addresses (e.g., physical IP addresses) are associated with components in a physical network, and overlay addresses (e.g., overlay IP addresses) are associated with entities in a virtual or overlay network. A physical IP address is an IP address associated with a physical device (e.g., a network device) in the underlying or physical network. For example, each NVD has an associated physical IP address. An overlay IP address is an overlay address associated with an entity in an overlay network, such as an overlay address associated with a compute instance in a customer's Virtual Cloud Network (VCN). Two different customers or tenants (each with its own private VCN) may potentially use the same overlay IP address in their VCNs without knowing about each other. Both physical IP addresses and overlay IP addresses are types of real IP addresses. These are separate from virtual IP addresses. A virtual IP address is typically a single IP address that represents or maps to multiple real IP addresses. A virtual IP address provides a one-to-many mapping between virtual IP addresses and multiple real IP addresses. For example, a load balancer can use a VIP to map to or represent multiple servers, each with its own real IP address.

[0066] Cloud infrastructure, or CSPI, is physically hosted in one or more data centers in one or more regions around the world. CSPI may include components in the physical or underlying network and virtualized components (e.g., virtual networks, compute instances, virtual machines, etc.) in a virtual network built on top of the physical network components. In some embodiments, CSPI is organized and hosted in domains, regions, and availability domains. A region is typically a local geographic area containing one or more data centers. Regions are often independent of each other and can be geographically separated, for example, across countries or even continents. For example, one region might be in Australia, another in Japan, yet another in India, and so on. CSPI resources are partitioned between regions, such that each region has its own independent subset of CSPI resources. Each region can provide a set of core infrastructure services and resources, such as compute resources (e.g., bare metal servers, virtual machines, containers, and related infrastructure); storage resources (e.g., block volume storage, file storage, object storage, archive storage); networking resources (e.g., virtual cloud networks (VCNs), load balancing resources, connectivity to on-premises networks); database resources; edge networking resources (e.g., DNS); and access management and monitoring resources, etc. Each region typically has multiple paths connecting it to other regions within the domain.

[0067] Generally, applications are deployed in the areas where they are used most frequently (i.e., on the infrastructure associated with that area) because using nearby resources is faster than using resources far away. Applications can also be deployed in different areas for various reasons, such as redundancy to mitigate the risk of regional events such as large weather systems or earthquakes, or to meet the requirements of changing legal jurisdictions, tax domains, and other business or social standards.

[0068] Data centers within a region can be further organized and subdivided into Availability Domains (ADs). An Availability Domain can correspond to one or more data centers located within a region. A region can consist of one or more Availability Domains. In such a distributed environment, CSPI resources are either region-specific (such as Virtual Cloud Networks (VCNs)) or Availability Domain-specific (such as compute instances).

[0069] Availability Zones (ADs) within a region are isolated and fault-tolerant, and configured to make simultaneous failures unlikely. This is achieved by ensuring that ADs do not share critical infrastructure resources (such as networking, physical cabling, cable paths, cable entry points, etc.), making a failure at one AD within a region unlikely to affect the availability of other ADs in the same region. ADs within the same region can be interconnected via low-latency, high-bandwidth networks, making it possible to provide high-availability connectivity to other networks (e.g., the internet, customer on-premises networks, etc.) and to build replication systems across multiple ADs for both high availability and disaster recovery. Cloud services use multiple ADs to ensure high availability and prevent resource failures. As the infrastructure provided by IaaS providers grows, additional capacity can be added to more regions and ADs. Traffic between availability domains is typically encrypted.

[0070] In some embodiments, regions are grouped into domains. A domain is a logical collection of regions. Domains are isolated from each other and do not share any data. Regions within the same domain can communicate with each other, but regions in different domains cannot. A customer's lease or account with a CSP exists within a single domain and can be distributed across one or more regions belonging to that domain. Typically, when a customer subscribes to an IaaS service, the lease or account is created for that customer in the customer-designated region within a domain (referred to as the "home" region). A customer can extend their lease across one or more other regions within a domain. A customer cannot access regions that are not within the domain where their lease exists.

[0071] IaaS providers can offer multiple domains, each catering to a specific set of customers or users. For example, a business domain can be offered to business customers. As another example, a country-specific domain can be offered to customers within a specific country. And yet another example, a government domain can be offered to governments. For instance, a government domain can cater to a specific government and may offer a higher level of security compared to the business domain. For example, Oracle Cloud Infrastructure (OCI) currently offers one domain for its business region and two domains (e.g., FedRAMP license and IL5 license) for its government cloud region.

[0072] In some embodiments, an Active Directory (AD) can be subdivided into one or more fault domains. A fault domain is a grouping of infrastructure resources within an AD to provide anti-affinity. Fault domains allow for the distribution of compute instances so that instances are not on the same physical hardware within a single AD. This is called anti-affinity. A fault domain refers to a group of hardware components (computers, switches, etc.) that share a single point of failure. Compute pools are logically divided into fault domains. Therefore, a hardware failure or compute hardware maintenance event affecting one fault domain does not affect instances in other fault domains. Depending on the embodiment, the number of fault domains per AD can vary. For example, in some embodiments, each AD contains three fault domains. Fault domains act as logical data centers within the AD.

[0073] When a customer subscribes to IaaS services, resources from CSPI are provisioned to the customer and associated with the customer's lease. Customers can use these provisioned resources to build private networks and deploy resources on those networks. Customer networks hosted in the cloud by CSPI are called Virtual Cloud Networks (VCNs). Customers can use the CSPI resources allocated to them to set up one or more VCNs. A VCN is a virtual or software-defined private network. Customer resources deployed in a customer's VCN can include compute instances (e.g., virtual machines, bare metal instances) and other resources. These compute instances can represent various customer workloads, such as applications, load balancers, databases, etc. Compute instances deployed on a VCN can communicate with publicly accessible endpoints (“public endpoints”) via public networks (such as the Internet), with other instances in the same VCN or other VCNs (e.g., other VCNs belonging to the customer or VCNs not belonging to the customer), with the customer's on-premises data center or network, and with service endpoints and other types of endpoints.

[0074] CSPs can use CSPIs to provide various services. In some instances, CSPI clients can act as service providers themselves and use CSPI resources to provide services. Service providers can expose service endpoints characterized by identifying information (e.g., IP address, DNS name, and port). Client resources (e.g., compute instances) can consume a specific service by accessing the service endpoints exposed by the service for that specific service. These service endpoints are typically publicly accessible to users via public communication networks (such as the Internet) using the public IP address associated with the endpoint. Publicly accessible network endpoints are sometimes referred to as public endpoints.

[0075] In some embodiments, a service provider may expose the service via an endpoint (sometimes referred to as a service endpoint). Clients of the service can then use this service endpoint to access the service. In some implementations, the service endpoint provided for the service can be accessed by multiple clients intending to consume the service. In other implementations, a dedicated service endpoint may be provided to a client, so that only that client can use that dedicated service endpoint to access the service.

[0076] In some embodiments, when a VCN is created, it is associated with a Private Overlay Classless Inter-Domain Routing (CIDR) address space, which refers to a range of private overlay IP addresses (e.g., 10.0 / 16) assigned to the VCN. A VCN includes associated subnets, routing tables, and gateways. A VCN resides within a single area but can span one or more or all availability domains within that area. A gateway is a virtual interface configured for the VCN and enables communication to and from the VCN to one or more endpoints outside the VCN. One or more different types of gateways can be configured for the VCN to enable communication to or from different types of endpoints.

[0077] A VCN can be subdivided into one or more subnets, such as one or more subnets. Therefore, a subnet is a configuration unit or subdivision that can be created within a VCN. A VCN can have one or more subnets. Each subnet within a VCN is associated with a contiguous range of overlay IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24) that do not overlap with other subnets within that VCN or subnets representing subsets of the address space within the VCN's address space.

[0078] Each compute instance is associated with a Virtual Network Interface Card (VNIC), which enables the compute instance to join a subnet within a VCN. A VNIC is the logical representation of a physical network interface card (NIC). Generally, a VNIC is the interface between an entity (e.g., a compute instance, a service) and a virtual network. A VNIC exists within a subnet and has one or more associated IP addresses and associated security rules or policies. A VNIC is equivalent to a Layer-2 port on a switch. A VNIC is attached to both the compute instance and the subnet within the VCN. The VNIC associated with a compute instance enables the compute instance to become part of a VCN's subnet and allows the compute instance to communicate with endpoints such as those on the same subnet, those in different subnets within the VCN, or those outside the VCN. Therefore, the VNIC associated with a compute instance determines how the compute instance connects to endpoints inside and outside the VCN. When a compute instance is created and added to a subnet within a VCN, a VNIC for the compute instance is created and associated with that compute instance. For a subnet that includes a set of compute instances, the subnet contains a VNIC corresponding to that set of compute instances, and each VNIC is attached to a compute instance within that set of compute instances.

[0079] Each compute instance is assigned a private overlay IP address via a VNIC associated with it. This private overlay IP address is assigned to the VNIC associated with the compute instance when the compute instance is created and used to route traffic to and from the compute instance. All VNICs within a given subnet use the same routing table, security list, and DHCP options. As described above, each subnet within a VCN is associated with a contiguous range of overlay IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24) that do not overlap with other subnets within that VCN or subnets representing subsets of the address space within the VCN's address space. For a VNIC on a specific subnet of a VCN, the private overlay IP address assigned to the VNIC is an address derived from the contiguous range of overlay IP addresses allocated to that subnet.

[0080] In some embodiments, in addition to a private overlay IP address, a compute instance may optionally be assigned additional overlay IP addresses, such as one or more public IP addresses, for example, if in a public subnet. These multiple addresses are assigned on the same VNIC or on multiple VNICs associated with the compute instance. However, each instance has a primary VNIC created during instance startup and associated with the overlay private IP address assigned to the instance—this primary VNIC cannot be removed. Additional VNICs (referred to as secondary VNICs) may be added to an existing instance in the same availability domain as the primary VNIC. All VNICs are in the same availability domain as the instance. Secondary VNICs may be in a subnet within the same VCN as the primary VNIC, or in a different subnet within the same VCN or different VCNs.

[0081] If a compute instance is in a public subnet, it can be optionally assigned a public IP address. When a subnet is created, it can be specified as either a public or private subnet. A private subnet means that resources within the subnet (e.g., compute instances) and associated VNICs cannot have public overriding IP addresses. A public subnet means that resources within the subnet and associated VNICs can have public IP addresses. Customers can specify that a subnet exists within a single availability domain or across multiple availability domains across regions or domains.

[0082] As described above, a VCN can be subdivided into one or more subnets. In some embodiments, a virtual router (VR) configured for the VCN (referred to as a VCN VR or VR-only) enables communication between the VCN's subnets. For a subnet within the VCN, the VR represents the logical gateway for that subnet, enabling that subnet (i.e., compute instances on that subnet) to communicate with endpoints on other subnets within the VCN as well as with other endpoints outside the VCN. The VCN VR is a logical entity configured to route traffic between the VNIC within the VCN and the virtual gateway (“gateway”) associated with the VCN. The following section discusses… Figure 1Further description of the gateway. A VCN VR is a Layer-3 / IP layer concept. In one embodiment, there exists a VCN VR for a VCN, where the VCN VR potentially has an unlimited number of ports addressed by IP addresses, with one port for each subnet of the VCN. In this way, the VCN VR has a different IP address for each subnet within the VCN to which the VCN VR is attached. The VR also connects to various gateways configured for the VCN. In some embodiments, a specific overlay IP address from the overlay IP address range of the subnet is reserved for the port of the VCN VR for that subnet. For example, consider a VCN with two subnets, which have associated address ranges 10.0 / 16 and 10.1 / 16, respectively. For a first subnet within the VCN with an address range of 10.0 / 16, addresses from this range are reserved for the port of the VCN VR for that subnet. In some instances, the first IP address from the range may be reserved for the VCN VR. For example, for a subnet covering the IP address range 10.0 / 16, the IP address 10.0.0.1 can be reserved for a port on the VCN VR for that subnet. For a second subnet within the same VCN with an address range of 10.1 / 16, the VCN VR can have a port on the second subnet with the IP address 10.1.0.1. The VCN VR has a different IP address for each subnet within the VCN.

[0083] In some other embodiments, each subnet within a VCN may have its own associated VR, which can be addressed by the subnet using a reserved or default IP address associated with the VR. The reserved or default IP address may, for example, be a first IP address from a range of IP addresses associated with the subnet. The VNICs within the subnet can communicate (e.g., send and receive packets) with the VR associated with the subnet using the default or reserved IP address. In some such embodiments, the VR is the ingress / egress point of the subnet. The VR associated with a subnet within the VCN can communicate with other VRs associated with other subnets within the VCN. The VR can also communicate with a gateway associated with the VCN. The VR functionality of the subnet is running on or performed by one or more NVDs that perform the VNIC functionality of the VNICs within the subnet.

[0084] You can configure routing tables, security rules, and DHCP options for a VCN. The routing table is a virtual routing table for the VCN and contains rules that route traffic from subnets within the VCN to destinations outside the VCN via gateways or specially configured instances. The VCN's routing table can be customized to control how packets are forwarded / routed to or from the VCN. DHCP options refer to configuration information automatically provided to the instance when it boots up.

[0085] Security rules configured for a VCN represent the VCN's overriding firewall rules. Security rules can include ingress and egress rules and specify the types of traffic allowed to enter and exit instances within the VCN (e.g., based on protocol and port). Clients can choose whether a given rule is stateful or stateless. For example, a client can allow incoming SSH traffic from anywhere to a set of instances by establishing a stateful ingress rule using source CIDR 0.0.0.0 / 0 and destination TCP port 22. Security rules can be implemented using network security groups or security lists. A network security group consists of a set of security rules that apply only to resources within that group. A security list, on the other hand, includes rules that apply to all resources in any subnet using that security list. A VCN can be provided with a default security list containing default security rules. DHCP options configured for a VCN provide configuration information that is automatically provided to instances within the VCN at instance boot time.

[0086] In some embodiments, the configuration information of a VCN is determined and stored by the VCN control plane. The VCN configuration information may include, for example, information about: the address range associated with the VCN, subnets and associated information within the VCN, one or more VRs associated with the VCN, compute instances and associated VNICs within the VCN, NVDs performing various virtualized network functions associated with the VCN (e.g., VNICs, VRs, gateways), VCN status information, and other VCN-related information. In some embodiments, a VCN distribution service publishes the configuration information or portions thereof stored by the VCN control plane to the NVD. The distributed information can be used to update information stored and used by the NVD (e.g., forwarding tables, routing tables, etc.) to forward packets to and from compute instances within the VCN.

[0087] In some embodiments, the creation of VCNs and subnets is handled by the VCN control plane (CP), and the initiation of compute instances is handled by the compute control plane. The compute control plane is responsible for allocating physical resources to the compute instances and then invoking the VCN control plane to create VNICs and attach them to the compute instances. The VCN CP also maps VCN data to the VCN data plane, which is configured to perform packet forwarding and routing functions. In some embodiments, the VCN CP provides a distribution service responsible for providing updates to the VCN data plane. Examples of VCN control planes are also available in... Figure 16 , Figure 17 , Figure 18 And depicted in Figure 19 (see reference numerals 1616, 1716, 1816 and 1916) and described below.

[0088] Customers can use resources hosted by CSPI to create one or more VCNs. Compute instances deployed on a customer's VCN can communicate with different endpoints. These endpoints can include CSPI-hosted endpoints and endpoints external to CSPI.

[0089] Various architectures are used to implement cloud-based services using CSPI. Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 , Figure 16 , Figure 17 , Figure 18 As depicted in Figure 20 and described below. Figure 1 This is a high-level diagram illustrating a distributed environment 100 of an overlay or client VCN hosted by CSPI, according to certain embodiments. Figure 1 The distributed environment described includes multiple components in the overlay network. Figure 1 The distributed environment 100 depicted herein is merely an example and is not intended to unduly limit the scope of the claimed embodiments. Many variations, substitutions, and modifications are possible. For example, in some implementations, Figure 1 The distributed environment described in the text can have more than Figure 1 The more or fewer systems or components shown can be combined into two or more systems, or can have different system configurations or arrangements.

[0090] like Figure 1 As illustrated in the example depicted, the distributed environment 100 includes a CSPI 101, which provides services and resources that customers can subscribe to and use to build their Virtual Cloud Network (VCN). In some embodiments, CSPI 101 provides IaaS services to subscribing customers. Data centers within CSPI 101 can be organized into one or more regions. An example region, “Region US” 102, is... Figure 1 As shown in the diagram, the customer has already configured customer VCN 104 for region 102. The customer can deploy various compute instances on VCN 104, which can include virtual machines or bare metal instances. Examples of instances include applications, databases, load balancers, etc.

[0091] exist Figure 1 In the embodiment depicted, customer VCN 104 includes two subnets, namely "Subnet-1" and "Subnet-2", each with its own CIDR IP address range. Figure 1 In this configuration, subnet-1 covers the IP address range of 10.0 / 16, and subnet-2 covers the address range of 10.1 / 16. VCN Virtual Router (VCN VR) 105 represents the logical gateway of the VCN, enabling communication between subnets of VCN 104 and with other endpoints outside the VCN. VCN VR 105 is configured to route traffic between the VNICs within VCN 104 and the gateway associated with VCN 104. VCN VR 105 provides a port for each subnet of VCN 104. For example, VCN VR 105 could provide a port with IP address 10.0.0.1 for subnet-1 and a port with IP address 10.1.0.1 for subnet-2.

[0092] Multiple compute instances can be deployed on each subnet, where compute instances can be virtual machine instances and / or bare metal instances. Compute instances within a subnet can be hosted by one or more host machines within CSPI 101. Compute instances join the subnet via a VNIC associated with the compute instance. For example, as... Figure 1 As shown, compute instance C1 is part of subnet-1 via the VNIC associated with it. Similarly, compute instance C2 is part of subnet-1 via the VNIC associated with it. In a similar manner, multiple compute instances (which can be virtual machine instances or bare metal instances) can be part of subnet-1. Each compute instance is assigned a private overlay IP address and MAC address via its associated VNIC. For example, in... Figure 1 In this subnet-1, compute instance C1 has an overlay IP address of 10.0.0.2 and a MAC address of M1, while compute instance C2 has a private overlay IP address of 10.0.0.3 and a MAC address of M2. Each compute instance in subnet-1 (including compute instances C1 and C2) has a default route to the VCN VR 105 using IP address 10.0.0.1, which is the IP address used by the VCN VR 105 for the port in subnet-1.

[0093] Subnet-2 can have multiple compute instances deployed on it, including virtual machine instances and / or bare metal instances. For example, as Figure 1 As shown, compute instances D1 and D2 are part of subnet-2 via the VNIC associated with the respective compute instance. Figure 1 In the embodiment depicted, compute instance D1 has an overlay IP address 10.1.0.2 and a MAC address MM1, while compute instance D2 has a private overlay IP address 10.1.0.3 and a MAC address MM2. Each compute instance in subnet-2 (including compute instances D1 and D2) has a default route to VCN VR 105 using IP address 10.1.0.1, which is the IP address used for the port of VCN VR 105 for subnet-2.

[0094] VCN A 104 may also include one or more load balancers. For example, a load balancer may be provided for a subnet and can be configured to load balance traffic across multiple compute instances on the subnet. A load balancer may also be provided to load balance traffic across subnets in a VCN.

[0095] A specific compute instance deployed on VCN 104 can communicate with a variety of different endpoints. These endpoints can include endpoints hosted by CSPI 200 and endpoints outside of CSPI 200. Endpoints hosted by CSPI 101 can include: endpoints on the same subnet as the specific compute instance (e.g., communication between two compute instances in subnet-1); endpoints on different subnets but within the same VCN (e.g., communication between a compute instance in subnet-1 and a compute instance in subnet-2); endpoints in different VCNs within the same region (e.g., communication between a compute instance in subnet-1 and an endpoint in a VCN within the same region 106 or 110, or communication between a compute instance in subnet-1 and an endpoint in service point 110 within the same region); or endpoints in VCNs in different regions (e.g., communication between a compute instance in subnet-1 and an endpoint in a VCN within a different region 108). Compute instances in subnets hosted by CSPI 101 can also communicate with endpoints not hosted by CSPI 101 (i.e., outside of CSPI 101). These external endpoints include endpoints in the customer’s on-premises network 116, endpoints in other remote cloud-hosted networks 118, public endpoints 114 that are accessible via public networks such as the Internet, and other endpoints.

[0096] VNICs associated with the source and destination compute instances are used to facilitate communication between compute instances on the same subnet. For example, compute instance C1 in subnet-1 might want to send packets to compute instance C2 in subnet-1. For packets originating from the source compute instance and destined for another compute instance in the same subnet, the packets are first processed by the VNIC associated with the source compute instance. The processing performed by the VNIC associated with the source compute instance may include determining the packet's destination information from the packet header, identifying any policies (e.g., security lists) configured for the VNIC associated with the source compute instance, determining the packet's next hop, performing any packet encapsulation / decapsulation functions as needed, and then forwarding / routing the packet to the next hop to facilitate communication to its intended destination. When the destination compute instance is in the same subnet as the source compute instance, the VNIC associated with the source compute instance is configured to identify the VNIC associated with the destination compute instance and forward packets to that VNIC for processing. The VNIC associated with the destination compute instance then performs the processing and forwards the packets to the destination compute instance.

[0097] For packets to be communicated from compute instances in a subnet to endpoints in different subnets within the same VCN, communication is facilitated by the VNIC associated with the source and destination compute instances, as well as the VCN VR. For example, if Figure 1 If compute instance C1 in subnet-1 wants to send a packet to compute instance D1 in subnet-2, the packet is first processed by the VNIC associated with compute instance C1. The VNIC associated with compute instance C1 is configured to route the packet to VCN VR 105 using the default route or port 10.0.0.1 of the VCN VR. VCN VR 105 is configured to route the packet to subnet-2 using port 10.1.0.1. The packet is then received and processed by the VNIC associated with D1, and the VNIC forwards the packet to compute instance D1.

[0098] For packets to be communicated from compute instances within VCN 104 to endpoints outside VCN 104, communication is facilitated by the VNIC associated with the source compute instance, VCN VR 105, and the gateway associated with VCN 104. One or more types of gateways can be associated with VCN 104. A gateway is an interface between a VCN and another endpoint located outside the VCN. A gateway is a Layer 3 / IP concept and enables a VCN to communicate with endpoints outside the VCN. Therefore, a gateway facilitates traffic flow between a VCN and other VCNs or networks. Various different types of gateways can be configured for a VCN to facilitate different types of communication with different types of endpoints. Depending on the gateway, communication can be conducted over a public network (e.g., the Internet) or over a private network. Various communication protocols can be used for these communications.

[0099] For example, compute instance C1 might want to communicate with an endpoint outside of VCN 104. The packet can first be processed by the VNIC associated with the source compute instance C1. The VNIC processing determines that the packet's destination is outside subnet-1 of C1. The VNIC associated with C1 can then forward the packet to VCN VR 105 of VCN 104. VCN VR 105 then processes the packet and, as part of the processing, determines a specific gateway associated with VCN 104 as the packet's next hop based on the packet's destination. VCN VR 105 can then forward the packet to the specifically identified gateway. For example, if the destination is an endpoint within the customer's on-premises network, the packet can be forwarded by VCN VR 105 to Dynamic Routing Gateway (DRG) gateway 122 configured for VCN 104. The packet can then be forwarded from the gateway to the next hop to facilitate communication to its final intended destination.

[0100] Various types of gateways can be configured for a VCN. Examples of gateways that can be configured for a VCN are available in [link / details]. Figure 1 It is depicted in the image and described below. An example of a gateway associated with a VCN is also shown below. Figure 16 , Figure 17 , Figure 18 The gateways depicted in Figure 19 (e.g., referenced by reference numerals 1634, 1636, 1638, 1734, 1736, 1738, 1834, 1836, 1838, 1934, 1936 and 1938) and described below. Figure 1As illustrated in the embodiments depicted, a Dynamic Routing Gateway (DRG) 122 can be added to or associated with a customer VCN 104 and provides a path for private network traffic communication between the customer VCN 104 and another endpoint, which can be the customer's on-premises network 116, a VCN 108 in a different region of CSPI 101, or another remote cloud network 118 not hosted by CSPI 101. The customer's on-premises network 116 can be a customer network or customer data center built using the customer's resources. Access to the customer's on-premises network 116 is typically very restricted. For a customer with both a customer's on-premises network 116 and one or more VCNs 104 deployed or hosted in the cloud by CSPI 101, the customer may want their on-premises network 116 and their cloud-based VCN 104 to be able to communicate with each other. This allows the customer to build an extended hybrid environment encompassing the customer's VCN 104 hosted by CSPI 101 and their on-premises network 116. The DRG 122 enables such communication. To enable such communication, a communication channel 124 is established, with one endpoint in the customer's local deployment network 116 and the other endpoint in CSPI 101 connected to the customer's VCN 104. Communication channel 124 can be over a public or private communication network, such as the Internet. Various communication protocols can be used, such as IPsec VPN technology over a public communication network (such as the Internet), Oracle's FastConnect technology using a private network instead of a public network, and so on. The device or equipment in the customer's local deployment network 116 forming one endpoint of communication channel 124 is called a Customer Premises Equipment (CPE), such as... Figure 1 The CPE 126 is depicted in the diagram. On the CSPI101 side, the endpoint can be the host machine executing DRG 122.

[0101] In some embodiments, a remote peering connection (RPC) can be added to the DRG, which allows a customer to peer one VCN with another VCN in a different region. Using such an RPC, customer VCN 104 can use DRG 122 to connect to VCN 108 in another region. DRG 122 can also be used to communicate with other remote cloud networks 118 that are not hosted by CSPI 101, such as Microsoft Azure Cloud, Amazon AWS Cloud, etc.

[0102] like Figure 1As shown, an Internet Gateway (IGW) 120 can be configured for a client VCN 104, enabling compute instances on VCN 104 to communicate with a public endpoint 114 accessible via a public network such as the Internet. The IGW 120 is a gateway connecting the VCN to a public network such as the Internet. The IGW 120 enables public subnets within the VCN, such as VCN 104 (where resources within the public subnet have publicly overriding IP addresses), to directly access the public endpoint 112 on the public network 114, such as the Internet. Using the IGW 120, connections can originate from subnets within VCN 104 or from the Internet.

[0103] Network Address Translation (NAT) gateway 128 can be configured for a customer's VCN 104 and enable cloud resources within the customer's VCN that do not have private public overlay IP addresses to access the Internet, and NAT gateway 128 does so without exposing those resources to direct inbound Internet connections (e.g., L4-L7 connections). This allows private subnets within the VCN (such as private subnet-1 in VCN 104) to have private access to public endpoints on the Internet. In the NAT gateway, connections can only be initiated from private subnets to the public Internet, and not from the Internet to private subnets.

[0104] In some embodiments, a Service Gateway (SGW) 126 may be configured for a client VCN 104 and provide a path for private network traffic between VCN 104 and supported service endpoints in service network 110. In some embodiments, service network 110 may be provided by a CSP and may offer a variety of services. An example of such a service network is Oracle's service network, which provides a variety of services that can be used by clients. For example, compute instances (e.g., database systems) in a private subnet of client VCN 104 may back up data to service endpoints (e.g., object storage) without requiring a public IP address or access to the Internet. In some embodiments, a VCN may have only one SGW, and connections may only originate from subnets within the VCN and not from service network 110. If a VCN peers to another VCN, resources in the other VCN typically cannot access the SGW. Resources in an on-premises network connected to a VCN using FastConnect or VPN Connect may also use the service gateway configured for that VCN.

[0105] In some implementations, SGW 126 uses the concept of a Service Classless Inter-Domain Routing (CIDR) label, which is a string representing all regional ranges of public IP addresses for the service or group of services of interest. Customers use service CIDR labels to control traffic to services when configuring SGW and related routing rules. Customers can optionally use service CIDR labels when configuring security rules to avoid having to adjust them if the public IP addresses of services change in the future.

[0106] Local peering gateway (LPG) 132 is a gateway that can be added to customer VCN 104 and enable VCN 104 to peer with another VCN in the same area. Peering means that VCNs communicate using private IP addresses without traffic traversing public networks such as the Internet or routing traffic through the customer's locally deployed network 116. In a preferred embodiment, a VCN has a separate LPG for each peer it establishes. Local peering, or VCN peering, is a common practice for establishing network connectivity between different applications or infrastructure management functions.

[0107] Service providers (such as service providers in service network 110) can use different access models to provide access to services. Under the public access model, services can be exposed as public endpoints, publicly accessible by compute instances in a customer's VCN via public networks such as the Internet and / or privately accessible via SGW 126. Under a specific private access model, services can be accessed as private IP endpoints within a private subnet of the customer's VCN. This is called private endpoint (PE) access and allows service providers to expose their services as instances within the customer's private network. A private endpoint resource represents a service within the customer's VCN. Each PE is represented as a VNIC (referred to as a PE-VNIC, with one or more private IPs) within a subnet selected by the customer within the customer's VCN. Thus, a PE provides a method of presenting a service within a private customer VCN subnet using a VNIC. Because the endpoint is exposed as a VNIC, all characteristics associated with the VNIC (such as routing rules, security lists, etc.) are now available to the PE VNIC.

[0108] Service providers can register their services to make them accessible via a PE (Private Provider). Providers can associate policies with services, which restricts the visibility of services to customer leases. Providers can register multiple services under a single Virtual IP address (VIP), especially for multi-tenant services. Multiple such private endpoints (across multiple VCNs) can exist representing the same service.

[0109] Compute instances in a private subnet can then access the service using the private IP address or service DNS name of the PE VNIC. Compute instances in a customer VCN can access the service by sending traffic to the private IP address of the PE in the customer VCN. A Private Access Gateway (PAGW) 130 is a gateway resource that can be attached to a service provider VCN (e.g., a VCN in service network 110), acting as the ingress / egress point for all traffic originating from / to the private endpoint of the customer subnet. PAGW 130 allows providers to scale the number of PE connections without utilizing their internal IP address resources. A provider only needs to configure one PAGW for any number of services registered in a single VCN. A provider can represent a service as a private endpoint in multiple VCNs of one or more customers. From the customer's perspective, the PE VNIC is not an instance attached to the customer, but rather appears to be attached to the service the customer wishes to interact with. Traffic destined for the private endpoint is routed to the service via PAGW 130. These are referred to as customer-to-service private connections (C2S connections).

[0110] The PE concept can also be used to extend private access to services to a customer's on-premises network and data center by allowing traffic to flow through FastConnect / IPsec links and private endpoints within the customer's VCN. Private access to services can also be extended to the customer's peering VCN by allowing traffic to flow between LPG 132 and the PE within the customer's VCN.

[0111] Customers can control routing within a VCN at the subnet level, allowing them to specify which subnets within a customer's VCN (such as VCN104) use each gateway. The VCN's routing table determines whether traffic is allowed to leave the VCN via a specific gateway. For example, in a given instance, the routing table for the public subnet within customer VCN 104 might allow non-local traffic to be sent via IGW 120. The routing table for the private subnet within the same customer VCN 104 might allow traffic destined for CSP services to be sent via SGW 126. All remaining traffic can be sent via NAT gateway 128. The routing table only controls traffic leaving the VCN.

[0112] Security lists associated with a VCN are used to control traffic entering the VCN via a gateway through inbound connections. All resources within a subnet use the same routing tables and security lists. Security lists can be used to control specific types of traffic allowed to enter or leave instances within a VCN subnet. Security list rules can include inbound and outbound rules. For example, inbound rules can specify allowed ranges of source addresses, while outbound rules can specify allowed ranges of destination addresses. Security rules can specify specific protocols (e.g., TCP, ICMP), specific ports (e.g., 22 for SSH, 3389 for Windows RDP), etc. In some implementations, the instance's operating system can enforce its own firewall rules consistent with the security list rules. Rules can be stateful (e.g., connections are tracked and responses are automatically allowed in the absence of explicit security list rules for response traffic) or stateless.

[0113] Access from a customer's VCN (i.e., through resources or compute instances deployed on VCN 104) can be categorized as public access, private access, or dedicated access. Public access refers to an access model where a public IP address or NAT is used to access a public endpoint. Private access enables customer workloads (e.g., resources in a private subnet) within VCN 104 with private IP addresses to access services without traversing a public network such as the Internet. In some embodiments, CSPI 101 enables customer VCN workloads with private IP addresses to access the service (its public service endpoint) using a service gateway. Thus, the service gateway provides a private access model by establishing a virtual link between the customer's VCN and the public endpoint of the service residing outside the customer's private network.

[0114] Additionally, CSPI can use technologies such as FastConnect public peering to provide dedicated public access, where customer-locally deployed instances can use FastConnect connections to access one or more services within the customer's VCN without traversing public networks such as the Internet. CSPI can also use FastConnect private peering to provide dedicated private access, where customer-locally deployed instances with private IP addresses can use FastConnect connections to access the customer's VCN workloads. FastConnect is an alternative to using the public Internet to connect a customer's local network to CSPI and its services for network connectivity. Compared to Internet-based connections, FastConnect provides a simple, resilient, and cost-effective way to create dedicated and private connections with higher bandwidth options and a more reliable and consistent networking experience.

[0115] Figure 1The description attached above illustrates the various virtualization components in the example virtual network. As mentioned above, the virtual network is built on top of the underlying physical or base network. Figure 2 A simplified architecture diagram is depicted within the physical network of a CSPI 200, according to certain embodiments, illustrating the physical components that provide the underlying physical network for the virtual network. As shown, CSPI 200 provides a distributed environment including components and resources (e.g., compute, memory, and networking resources) provided by a cloud service provider (CSP). These components and resources are used to provide cloud services (e.g., IaaS services) to subscribers (i.e., customers who have already subscribed to one or more services provided by the CSP). Based on the services subscribed to by the customer, a subset of the resources of CSPI 200 (e.g., compute, memory, and networking resources) is provisioned to the customer. The customer can then use the physical compute, memory, and networking resources provided by CSPI 200 to build their own customizable and private cloud-based (i.e., CSPI-hosted) virtual networks. As previously indicated, these customer networks are referred to as Virtual Cloud Networks (VCNs). Customers can deploy one or more customer resources, such as compute instances, on these customer VCNs. Compute instances can take the form of virtual machines, bare metal instances, etc. CSPI 200 provides infrastructure and a complementary set of cloud services that enable customers to build and run a wide range of applications and services in a highly available managed environment.

[0116] exist Figure 2 In the example embodiment depicted, the physical components of CSPI 200 include one or more physical host machines or physical servers (e.g., 202, 206, 208), network virtualization devices (NVDs) (e.g., 210, 212), top-of-rack (TOR) switches (e.g., 214, 216), and a physical network (e.g., 218), as well as switches within physical network 218. The physical host machines or servers can host and execute various compute instances that join one or more subnets of the VCN. Compute instances can include virtual machine instances and bare metal instances. For example, Figure 1 The various computational examples described in the text can be derived from... Figure 2 The physical host machine described in the diagram is used for hosting virtual machine compute instances in a VCN. Virtual machine compute instances in a VCN can be executed by a single host machine or by multiple different host machines. Physical host machines can also host virtual host machines, container-based host machines, or other functionalities. Figure 1 The VNIC and VCN VR described in the text can be generated by Figure 2 The NVD execution described in the text. Figure 1 The gateway described in the text can be... Figure 2 The host machine and / or NVD execution described in the text.

[0117] A host machine or server can execute a hypervisor (also known as a virtual machine monitor or VMM) that creates and enables virtualized environments on the host machine. Virtualization or virtualized environments facilitate cloud-based computing. One or more compute instances can be created, executed, and managed on the host machine by a hypervisor on that host machine. The hypervisor on the host machine enables the host machine's physical computing resources (e.g., compute, memory, and networking resources) to be shared among various compute instances executed by the host machine.

[0118] For example, such as Figure 2 As depicted, host machines 202 and 208 execute hypervisors 260 and 266, respectively. These hypervisors can be implemented using software, firmware, or hardware, or a combination thereof. Typically, a hypervisor is a process or software layer residing above the host machine's operating system (OS), which in turn executes on the host machine's hardware processor. Hypervisors provide a virtualization environment by enabling the host machine's physical computing resources (e.g., processing resources such as processors / cores, memory resources, networking resources) to be shared among various virtual machine computing instances executed by the host machine. For example, in... Figure 2 In this configuration, hypervisor 260 can reside on the operating system of host machine 202 and enable the computing resources (e.g., processing, memory, and networking resources) of host machine 202 to be shared among computing instances (e.g., virtual machines) executed by host machine 202. Virtual machines can have their own operating systems (called guest operating systems), which may be the same as or different from the host machine's operating system. The operating system of a virtual machine executed on the host machine can be the same as or different from the operating system of another virtual machine executed on the same host machine. Therefore, the hypervisor enables multiple operating systems to execute simultaneously while sharing the same computing resources of the host machine. Figure 2 The host machines described in the text may have the same or different types of management programs.

[0119] A compute instance can be a virtual machine instance or a bare metal instance. Figure 2 In the diagram, compute instance 268 on host machine 202 and compute instance 274 on host machine 208 are examples of virtual machine instances. Host machine 206 is an example of a bare metal instance provided to a customer.

[0120] In some instances, an entire host machine can be provisioned to a single customer, and all compute instances (virtual machines or bare metal instances) hosted by that host machine belong to that same customer. In other instances, the host machine can be shared among multiple customers (i.e., multiple tenants). In such multi-tenant scenarios, the host machine can host virtual machine compute instances belonging to different customers. These compute instances can be members of different VCNs for different customers. In some embodiments, bare metal compute instances are hosted by bare metal servers without a hypervisor. When bare metal compute instances are provisioned, a single customer or tenant maintains control over the physical CPU, memory, and network interfaces of the host machine hosting the bare metal instance, and the host machine is not shared with other customers or tenants.

[0121] As previously described, each compute instance, as part of a VCN, is associated with a VNIC that enables the compute instance to become a member of a subnet of the VCN. The VNIC associated with a compute instance facilitates communication to and from the compute instance for packets or frames. A VNIC is associated with a compute instance when it is created. In some embodiments, for compute instances executed by a host machine, the VNIC associated with that compute instance is executed by an NVD connected to the host machine. For example, in Figure 2 In this example, host machine 202 executes a virtual machine compute instance 268 associated with VNIC 276, and VNIC 276 is executed by NVD 210 connected to host machine 202. As another example, a bare metal instance 272 hosted by host machine 206 is associated with VNIC 280 executed by NVD 212 connected to host machine 206. As yet another example, VNIC 284 is associated with compute instance 274 executed by host machine 208, and VNIC 284 is executed by NVD 212 connected to host machine 208.

[0122] For compute instances hosted by a host machine, NVDs connected to that host machine also execute VCN VR corresponding to the VCN where the compute instance is a member. For example, in Figure 2 In the embodiment depicted, NVD 210 executes VCN VR 277 corresponding to the VCN of compute instance 268, which is a member of the VCN. NVD 212 may also execute one or more VCN VR 283 corresponding to the VCNs of compute instances hosted by host machines 206 and 208.

[0123] The host machine may include one or more network interface cards (NICs) that enable the host machine to connect to other devices. The NIC on the host machine may provide one or more ports (or interfaces) that allow the host machine to communicatively connect to another device. For example, the host machine may use one or more ports (or interfaces) provided on the host machine and the NVD to connect to the NVD. The host machine may also connect to other devices, such as another host machine.

[0124] For example, in Figure 2 In this configuration, host machine 202 connects to NVD 210 via link 220, which extends between port 234 provided by NIC 232 of host machine 202 and port 236 of NVD 210. Host machine 206 connects to NVD 212 via link 224, which extends between port 246 provided by NIC 244 of host machine 206 and port 248 of NVD 212. Host machine 208 connects to NVD 212 via link 226, which extends between port 252 provided by NIC 250 of host machine 208 and port 254 of NVD 212.

[0125] The NVD is then connected to a top-of-rack (TOR) switch via a communication link, which is connected to physical network 218 (also known as a switch architecture). In some embodiments, the links between the host machine and the NVD, and between the NVD and the TOR switch, are Ethernet links. For example, in Figure 2 In this configuration, NVDs 210 and 212 are connected to TOR switches 214 and 216 via links 228 and 230, respectively. In some embodiments, links 220, 224, 226, 228, and 230 are Ethernet links. The collection of host machines and NVDs connected to the TOR is sometimes referred to as a rack.

[0126] Physical network 218 provides a communication structure that enables TOR switches to communicate with each other. Physical network 218 can be a multi-level network. In some implementations, physical network 218 is a multi-level Clos network of switches, where TOR switches 214 and 216 represent leaf-level nodes of the multi-level and multi-node physical switching network 218. Different Clos network configurations are possible, including but not limited to 2-level networks, 3-level networks, 4-level networks, 5-level networks, and general "n"-level networks. Examples of Clos networks are provided in... Figure 5 It is depicted in the middle and described below.

[0127] Various connection configurations between the host machine and the NVD, such as one-to-one, many-to-one, and one-to-many configurations, are possible. In a one-to-one configuration implementation, each host machine connects to its own individual NVD. For example, in... Figure 2In this configuration, host machine 202 connects to NVD 210 via its NIC 232. In a many-to-one configuration, multiple host machines connect to a single NVD. For example, in... Figure 2 In this configuration, host machines 206 and 208 are connected to the same NVD 212 via NICs 244 and 250, respectively.

[0128] In a one-to-many configuration, a host machine connects to multiple NVDs. Figure 3 An example within the CSPI 300 is shown, where a host machine is connected to multiple NVDs. (Example follows) Figure 3 As shown, host machine 302 includes a network interface card (NIC) 304, which includes multiple ports 306 and 308. Host machine 300 is connected to a first NVD 310 via port 306 and link 320, and to a second NVD 312 via port 308 and link 322. Ports 306 and 308 may be Ethernet ports, and links 320 and 322 between host machine 302 and NVDs 310 and 312 may be Ethernet links. NVD 310 is then connected to a first TOR switch 314, and NVD 312 is connected to a second TOR switch 316. The links between NVDs 310 and 312 and TOR switches 314 and 316 may be Ethernet links. TOR switches 314 and 316 represent tier-0 switching devices in a multi-level physical network 318.

[0129] Figure 3 The layout depicted provides two separate physical network paths from physical switch network 318 to host machine 302: the first path traverses TOR switch 314 to NVD 310 and then to host machine 302, and the second path traverses TOR switch 316 to NVD 312 and then to host machine 302. These separate paths provide enhanced availability (referred to as high availability) for host machine 302. If one of the paths (e.g., a link in one of the paths breaks) or a device (e.g., a particular NVD is not running) experiences a problem, the other path can be used for communication with host machine 302.

[0130] exist Figure 3 In the configuration depicted, the host machine connects to two different NVDs using two different ports provided by the host machine's NIC. In other embodiments, the host machine may include multiple NICs enabling connectivity from the host machine to multiple NVDs.

[0131] Return to reference Figure 2An NVD is a physical device or component that performs one or more network and / or storage virtualization functions. An NVD can be any device with one or more processing units (e.g., CPU, Network Processing Unit (NPU), FPGA, packet processing pipeline, etc.), memory including cache, and ports. Various virtualization functions can be executed by software / firmware performed by one or more processing units of the NVD.

[0132] NVDs can be implemented in various different forms. For example, in some embodiments, an NVD is implemented as an interface card called a smartNIC or intelligent NIC with an embedded processor. A smartNIC is a device separate from the NIC on the host machine. Figure 2 In this context, NVD 210 and 212 can be implemented as smartNICs connected to host machine 202 and host machines 206 and 208, respectively.

[0133] However, smartNIC is just one example of an NVD implementation. Various other implementations are possible. For example, in some other implementations, the NVD, or one or more functions performed by the NVD, may be incorporated into or performed by one or more host machines, one or more TOR switches, and other components of the CSPI 200. For instance, the NVD may be embodied in a host machine, where the functions performed by the NVD are performed by the host machine. As another example, the NVD may be part of a TOR switch, or the TOR switch may be configured to perform functions performed by the NVD, enabling the TOR switch to perform various complex packet transformations for public clouds. TORs that perform NVD functions are sometimes referred to as smart TORs. In yet another implementation, where virtual machine (VM) instances, rather than bare metal (BM) instances, are provided to customers, the functions performed by the NVD may be implemented within the hypervisor of the host machine. In some other implementations, some of the functions of the NVD may be offloaded to a centralized service running on a series of host machines.

[0134] In some embodiments, such as when implemented as Figure 2 When a smartNIC is shown, an NVD can include multiple physical ports that enable the NVD to connect to one or more host machines and one or more TOR switches. Ports on an NVD can be categorized as host-facing ports (also known as "south ports") or network-facing or TOR-facing ports (also known as "north ports"). The host-facing ports of an NVD are used to connect the NVD to the host machine. Figure 2Examples of host-facing ports on the NVD 210 include port 236, and ports 248 and 254 on the NVD 212. Network-facing ports on the NVD are used to connect the NVD to a TOR switch. Figure 2 Examples of network-facing ports include port 256 on the NVD 210 and port 258 on the NVD 212. Figure 2 As shown, NVD 210 is connected to TOR switch 214 via link 228, which extends from port 256 of NVD 210 to TOR switch 214. Similarly, NVD 212 is connected to TOR switch 216 via link 230, which extends from port 258 of NVD 212 to TOR switch 216.

[0135] The NVD receives packets and frames from the host machine (e.g., packets and frames generated by compute instances hosted on the host machine) via its host-facing port, and after performing necessary packet processing, can forward the packets and frames to the TOR switch via its network-facing port. The NVD can also receive packets and frames from the TOR switch via its network-facing port, and after performing necessary packet processing, can forward the packets and frames to the host machine via its host-facing port.

[0136] In some embodiments, there can be multiple ports and associated links between the NVD and TOR switches. These ports and links can be aggregated to form a group of link aggregators (called a LAG) of multiple ports or links. Link aggregation allows multiple physical links between two endpoints (e.g., between the NVD and TOR switches) to be treated as a single logical link. All physical links in a given LAG can operate at the same speed in full-duplex mode. LAGs help improve the bandwidth and reliability of connections between two endpoints. If one physical link in the LAG fails, traffic is dynamically and transparently reassigned to one of the other physical links in the LAG. Aggregated physical links deliver higher bandwidth than each individual link. Multiple ports associated with an LAG are treated as a single logical port. Traffic can be load balanced across multiple physical links in the LAG. One or more LAGs can be configured between two endpoints. The two endpoints can be located between the NVD and TOR switches, between a host machine and the NVD, etc.

[0137] The NVD implements or performs network virtualization functions. These functions are performed by the software / firmware executed by the NVD. Examples of network virtualization functions include, but are not limited to: packet encapsulation and decapsulation functions; functions for creating VCN networks; functions for implementing network policies such as VCN security lists (firewalls); functions for facilitating the routing and forwarding of packets to and from compute instances in the VCN; and so on. In some embodiments, upon receiving a packet, the NVD is configured to perform a packet processing pipeline for processing the packet and determining how the packet is forwarded or routed. As part of this packet processing pipeline, the NVD may perform one or more virtual functions associated with the overlay network, such as performing VNICs associated with compute instances in the VCN, performing virtual routers (VRs) associated with the VCN, encapsulating and decapsulating packets to facilitate forwarding or routing in the virtual network, performing certain gateways (e.g., local peer gateways), security lists, network security groups, Network Address Translation (NAT) functions (e.g., public IP to private IP translation on a per-host-machine basis), throttling functions, and other functions.

[0138] In some embodiments, the packet processing data path in NVD may include multiple packet pipelines, each consisting of a series of packet transformation stages. In some implementations, upon receiving a packet, it is parsed and classified into a single pipeline. The packet is then processed linearly, stage by stage, until it is dropped or sent out through the NVD's interface. These stages provide basic functional packet processing building blocks (e.g., header verification, throttling, inserting new Layer-2 headers, implementing L4 firewalls, VCN encapsulation / decapsulation, etc.), allowing new pipelines to be constructed by combining existing stages, and new functionality to be added by creating new stages and inserting them into existing pipelines.

[0139] NVD can perform both control plane and data plane functions corresponding to those of VCN's control plane and data plane. An example of the VCN control plane is also available. Figure 16 , Figure 17 , Figure 18 And depicted in Figure 19 (see reference numerals 1616, 1716, 1816 and 1916) and described below. An example of the VCN data plane is shown in... Figure 16 , Figure 17 , Figure 18As depicted in Figure 19 (see reference numerals 1618, 1718, 1818, and 1918) and described below, the control plane functions include those for configuring how control data is forwarded on the network (e.g., establishing routes and routing tables, configuring VNICs, etc.). In some embodiments, a VCN control plane is provided that centrally computes all overlay mappings to the base layer and publishes them to the NVD and to virtual network edge devices such as various gateways (e.g., DRGs, SGWs, IGWs, etc.). Firewall rules can also be published using the same mechanism. In some embodiments, the NVD only acquires mappings associated with that NVD. The data plane functions include those for actual routing / forwarding of packets based on the configuration established using the control plane. The VCN data plane is implemented by encapsulating client network packets before they traverse the base layer network. Encapsulation / decapsulation functionality is implemented on the NVD. In some embodiments, the NVD is configured to intercept all network packets entering and leaving the host machine and perform network virtualization functions.

[0140] As indicated above, the NVD performs various virtualization functions, including VNICs and VCN VR. The NVD can execute VNICs associated with compute instances hosted by one or more host machines connected to the VNIC. For example, as... Figure 2 As depicted, NVD 210 performs the functions of VNIC 276 associated with compute instance 268 hosted by host machine 202 connected to NVD 210. As another example, NVD 212 performs VNIC 280 associated with bare-metal compute instance 272 hosted by host machine 206, and VNIC 284 associated with compute instance 274 hosted by host machine 208. Host machines can host compute instances belonging to different VCNs belonging to different clients, and NVDs connected to host machines can perform VNICs corresponding to compute instances (i.e., perform functions associated with the VNICs).

[0141] NVD also executes a VCN virtual router corresponding to the VCN of the compute instance. For example, in Figure 2 In the embodiments depicted, NVD 210 executes VCN VR 277 corresponding to the VCN to which compute instance 268 belongs. NVD 212 executes one or more VCN VR 283 corresponding to one or more VCNs to which the compute instances hosted by host machines 206 and 208 belong. In some embodiments, the VCN VR corresponding to that VCN is executed by all NVDs connected to the host machine hosting at least one compute instance belonging to that VCN. If the host machine hosts compute instances belonging to different VCNs, the NVDs connected to that host machine can execute VCN VR corresponding to those different VCNs.

[0142] In addition to VNIC and VCN VR, NVD can execute various software (e.g., daemons) and includes one or more hardware components that facilitate the various network virtualization functions performed by NVD. For simplicity, these individual components are grouped together as... Figure 2 The “packet processing component” is described in the diagram. For example, NVD 210 includes packet processing component 286, and NVD 212 includes packet processing component 288. For instance, the packet processing component of an NVD may include a packet processor configured to interact with the NVD’s ports and hardware interfaces to monitor all packets received by and used by the NVD and to store network information. Network information may include, for example, network flow information identifying different network flows handled by the NVD and per-flow information (e.g., per-flow statistics). In some embodiments, network flow information may be stored on a per-VNIC basis. The packet processor may perform packet-by-packet manipulation and implement stateful NAT and L4 firewall (FW). As another example, the packet processing component may include a replication agent configured to replicate information stored by the NVD to one or more different replication target stores. As yet another example, the packet processing component may include a logging agent configured to perform logging functions of the NVD. The packet processing component may also include software for monitoring the performance and health of the NVD and may also monitor the status and health of other components connected to the NVD.

[0143] Figure 1 The components of an example virtual or overlay network are shown, including a VCN, subnets within the VCN, compute instances deployed on the subnets, VNICs associated with the compute instances, the VCN's VR, and a set of gateways configured for the VCN. Figure 1 The overlay component described in the text can be made by Figure 2 One or more physical components described in the diagram are executed or managed. For example, a compute instance in a VCN can be executed or managed by... Figure 2 The VNIC described herein is executed or hosted by one or more host machines. For compute instances hosted by a host machine, the VNIC associated with that compute instance is typically executed by an NVD connected to that host machine (i.e., VNIC functionality is provided by an NVD connected to that host machine). The VCN VR functionality of a VCN is executed by all NVDs connected to the host machine hosting or executing a compute instance that is part of that VCN. Gateways associated with a VCN can be executed by one or more different types of NVDs. For example, some gateways can be executed by smartNICs, while others can be executed by one or more host machines or other implementations of NVDs.

[0144] As described above, compute instances in a client VCN can communicate with a variety of different endpoints, which can be within the same subnet as the source compute instance, in a different subnet but within the same VCN as the source compute instance, or with endpoints outside the source compute instance's VCN. These communications are facilitated using the VNIC, VCN VR, and gateway associated with the VCN, all of which are linked to the compute instance.

[0145] For communication between two compute instances on the same subnet within a VCN, this communication is facilitated using VNICs associated with the source and destination compute instances. The source and destination compute instances can be hosted on the same host machine or different host machines. Packets originating from the source compute instance can be forwarded from the host machine hosting the source compute instance to an NVD connected to that host machine. On the NVD, packets are processed using a packet processing pipeline that may include the execution of the VNIC associated with the source compute instance. Because the destination endpoint of the packet is within the same subnet, the execution of the VNIC associated with the source compute instance results in the packet being forwarded to the NVD executing the VNIC associated with the destination compute instance, which then processes the packet and forwards it to the destination compute instance. The VNICs associated with the source and destination compute instances can execute on the same NVD (e.g., when both the source and destination compute instances are hosted on the same host machine) or on different NVDs (e.g., when the source and destination compute instances are hosted on different host machines connected to different NVDs). The VNIC can use the routing / forwarding table stored in the NVD to determine the next hop for the packet.

[0146] For packets destined for endpoints in different subnets within the same VCN, the packet originating from the source compute instance is forwarded from the host machine hosting the source compute instance to the NVD connected to that host machine. On the NVD, a packet processing pipeline is used to process the packets, which may include the execution of one or more VNICs and the VR associated with the VCN. For example, as part of the packet processing pipeline, the NVD executes or invokes a function corresponding to the VNIC associated with the source compute instance (also known as executing the VNIC). The function executed by the VNIC may include viewing the VLAN tag on the packet. Since the packet's destination is outside the subnet, the VCN VR function is then invoked and executed by the NVD. The VCN VR then routes the packet to the NVD executing the VNIC associated with the destination compute instance. The VNIC associated with the destination compute instance then processes the packet and forwards it to the destination compute instance. The VNIC associated with the source and destination compute instances can run on the same NVD (e.g., when both the source and destination compute instances are hosted by the same host machine) or on different NVDs (e.g., when the source and destination compute instances are hosted by different host machines connected to different NVDs).

[0147] If the packet's destination is outside the source compute instance's VCN, the packet originating from the source compute instance is passed from the host machine hosting the source compute instance to the NVD connected to that host machine. The NVD executes the VNIC associated with the source compute instance. Since the packet's destination endpoint is outside the VCN, the packet is then processed by the VCN VR of that VCN. The NVD invokes VCNVR functionality, which allows the packet to be forwarded to the NVD executing the appropriate gateway associated with the VCN. For example, if the destination is an endpoint within the customer's on-premises network, the packet may be forwarded by the VCN VR to the NVD executing the DRG gateway configured for the VCN. The VCN VR may be executed on the same NVD as the NVD executing the VNIC associated with the source compute instance or by a different NVD. The gateway may be executed by an NVD, which can be a smartNIC, a host machine, or another NVD implementation. The packet is then processed by the gateway and forwarded to the next hop that facilitates communication to its intended destination endpoint. For example, in Figure 2 In the embodiment depicted, packets originating from compute instance 268 can be transmitted from host machine 202 to NVD 210 via link 220 (using NIC 232). On NVD 210, VNIC 276 is invoked because it is the VNIC associated with the source compute instance 268. VNIC 276 is configured to examine the encapsulation information in the packets and determine the next hop for forwarding the packets, with the aim of facilitating communication of the packets to their intended destination endpoint, and then forward the packets to the determined next hop.

[0148] Compute instances deployed on a VCN can communicate with a variety of different endpoints. These endpoints can include endpoints hosted by CSPI 200 and endpoints outside of CSPI 200. Endpoints hosted by CSPI 200 can include instances within the same VCN or other VCNs, which can be the customer's VCN or a VCN not belonging to the customer. Communication between endpoints hosted by CSPI 200 can be performed via physical network 218. Compute instances can also communicate with endpoints not hosted by CSPI 200 or outside of CSPI 200. Examples of these endpoints include endpoints within the customer's on-premises deployment network or data center, or public endpoints accessible via public networks such as the Internet. Communication with endpoints outside of CSPI 200 can use various communication protocols over public networks (e.g., the Internet). Figure 2 (not shown in the image) or private network ( Figure 2 (Not shown in the image) to execute.

[0149] Figure 2 The architecture of the CSPI 200 depicted herein is merely illustrative and not intended to be limiting. Variations, substitutions, and modifications are possible in alternative embodiments. For example, in some implementations, the CSPI 200 may have more advanced features than... Figure 2 The systems or components shown may include more or fewer systems or components, and may combine two or more systems, or may have different system configurations or arrangements. Figure 2 The systems, subsystems, and other components described herein are implemented using hardware or a combination thereof, with software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective systems. The software may be stored on a non-transitory storage medium (e.g., a memory device).

[0150] Figure 4 This describes, according to certain embodiments, a method for providing I / O virtualization to support connectivity between multi-tenant host machines and NVDs. For example... Figure 4 As depicted, host machine 402 executes a hypervisor 404 that provides a virtualization environment. Host machine 402 executes two virtual machine instances, VM1 406 belonging to customer / tenant #1 and VM2 408 belonging to customer / tenant #2. Host machine 402 includes a physical NIC 410 connected to NVD 412 via link 414. Each compute instance in the compute instances is attached to a VNIC executed by NVD 412. Figure 4 In one embodiment, VM1 406 is attached to VNIC-VM1 420 and VM2 408 is attached to VNIC-VM2 422.

[0151] like Figure 4As shown, NIC 410 includes two logical NICs, logical NIC A 416 and logical NIC B 418. Each virtual machine is attached to its own logical NIC and configured to work with its own logical NIC. For example, VM1 406 is attached to logical NIC A 416, and VM2 408 is attached to logical NIC B 418. Although host machine 402 includes only one physical NIC 410 shared by multiple tenants, each tenant's virtual machines believe they have their own host machine and NIC due to the logical NICs.

[0152] In some embodiments, each logical NIC is assigned its own VLAN ID. Thus, a specific VLAN ID is assigned to logical NIC A 416 for tenant #1, and a separate VLAN ID is assigned to logical NIC B 418 for tenant #2. When a packet is delivered from VM1 406, the tag assigned to tenant #1 is attached to the packet by the hypervisor, and the packet is then delivered from host machine 402 to NVD 412 via link 414. Similarly, when a packet is delivered from VM2 408, the tag assigned to tenant #2 is attached to the packet by the hypervisor, and the packet is then delivered from host machine 402 to NVD 412 via link 414. Therefore, the packet 424 delivered from host machine 402 to NVD 412 has an associated tag 426 that identifies the specific tenant and associated VM. On the NVD, for a packet 424 received from the host machine 402, a tag 426 associated with the packet is used to determine whether the packet should be processed by VNIC-VM1 420 or VNIC-VM2 422. The packet is then processed by the corresponding VNIC. Figure 4 The configuration described in the document allows each tenant's compute instance to believe that it owns its own host machine and NIC. Figure 4 The settings described in [the document] provide I / O virtualization for supporting multi-tenancy.

[0153] Figure 5 A simplified block diagram of a physical network 500 according to certain embodiments is depicted. Figure 5 The embodiments depicted are structured as Clos networks. A Clos network is a specific type of network topology designed to provide connectivity redundancy while maintaining high distributed bandwidth and maximum resource utilization. A Clos network is a non-blocking, multi-stage or multi-level switching network, wherein the number of stages or levels can be two, three, four, five, etc. Figure 5The embodiment depicted is a three-tier network including tiers 1, 2, and 3. TOR switch 504 represents a tier-0 switch in the Clos network. One or more NVDs are connected to the TOR switch. The tier-0 switch is also referred to as the edge device of the physical network. The tier-0 switch connects to a tier-1 switch, also referred to as a leaf switch layer. Figure 5 In the embodiment depicted, a set of n Layer-0 TOR switches is connected to a set of n Layer-1 switches, forming a pod. Each Layer-0 switch in a pod is interconnected with all Layer-1 switches in that pod, but there is no switch connectivity between pods. In some implementations, two pods are referred to as blocks. Each block is served by or connected to a set of n Layer-2 switches (sometimes called backbone switches). Several blocks may exist in the physical network topology. The Layer-2 switches are then connected to n Layer-3 switches (sometimes called super backbone switches). Packet communication on the physical network 500 is typically performed using one or more Layer-3 communication protocols. Typically, all layers of the physical network, except the TOR layer, are n-way redundant, allowing for high availability. Policies can be specified for pods and blocks to control the visibility of switches to each other in the physical network, enabling scaling of the physical network.

[0154] A key characteristic of Clos networks is that the maximum number of hops from one Layer-0 switch to another Layer-0 switch (or from an NVD connected to a Layer-0 switch to another NVD connected to a Layer-0 switch) is fixed. For example, in a Layer 3 Clos network, a maximum of seven hops are required for a packet to travel from one NVD to another, where the source and destination NVDs are connected to the leaf layers of the Clos network. Similarly, in a Layer 4 Clos network, a maximum of nine hops are required for a packet to travel from one NVD to another, where the source and destination NVDs are connected to the leaf layers of the Clos network. Therefore, the Clos network architecture maintains consistent latency throughout the network, which is important for communication within and between data centers. Clos topologies are horizontally scalable and cost-effective. Network bandwidth / throughput can be easily increased by adding more switches at each layer (e.g., more leaf and backbone switches) and by increasing the number of links between switches at adjacent layers.

[0155] In some embodiments, each resource within the CSPI is assigned a unique identifier called a Cloud Identifier (CID). This identifier is included as part of the resource's information and can be used to manage the resource, for example, via a console or API. An example syntax for a CID is:

[0156] ocid1.<RESOURCE TYPE> .<realm>[REGION][FUTURE USE].<UNIQUE ID>

[0157] in,

[0158] ocid1: A literal string indicating the CID version;

[0159] resource type: The type of resource (e.g., instance, volume, VCN, subnet, user, group, etc.);

[0160] realm: The realm where the resource resides. Example values ​​are "c1" for the commercial realm, "c2" for the government cloud realm, or "c3" for the federal government cloud realm, etc. Each realm can have its own domain name;

[0161] region: The region where the resource is located. This part can be empty if the region is not applicable to the resource;

[0162] future use: to be reserved for future use.

[0163] Unique ID: The unique part of the ID. The format may vary depending on the type of resource or service.

[0164] Cloud defense system

[0165] This describes techniques for monitoring and collecting data related to reverse or recursive DNS (rDNS) traffic associated with the monitored cloud environment.

[0166] A recursive Domain Name System (DNS) resolver can receive a fully qualified domain name (FQDN) as the subject of a DNS request from a requesting system, perform a lookup for the corresponding IP address (e.g., by using an authoritative DNS resolver), and then return the IP address to the requesting system in response to the request.

[0167] Compared to DNS requests, rDNS requests are reverse lookup requests. Therefore, upon receiving an rDNS request that includes an IP address, a recursive DNS resolver can perform a lookup for the corresponding FQDN within a set of pointers (PTR) records using the received IP address (e.g., via an authoritative DNS resolver), and then return the FQDN to the requesting system in response to the request. rDNS requests can be received from any system that wants to perform an FQDN lookup using an IP address. For example, a system might have already received an IP address as part of one or more packets received from the IP address, and therefore want to know the FQDN associated with that IP address (e.g., by trusting the FQDN to see if the IP address is trusted). As another example, if something is trying to connect to a host machine using SSH, it can generate a PTR query because OpenSSH has many conditions that might cause a call to getaddrinfo, which in turn causes the host to issue a PTR query.

[0168] As another example, rDNS requests can help the system determine whether a secondary system (e.g., a server) has set a PTR record, which can serve as a hint about the validity of packets received from the secondary system.

[0169] As another example, PTR records can help the system determine where network traffic originates (e.g., domain name).

[0170] rDNS traffic can include recursive DNS (rDNS) requests originating from the monitored environment (e.g., the monitored cloud environment) and responses to these requests received from DNS resolvers. This collected data can be analyzed to identify potential threats to the monitored environment. Potential threats can include actual threats as well as anomalous behavior within the monitored cloud environment. The collected data can be analyzed to identify potential sources of the monitored behavior (e.g., threats) and to identify one or more parts of the monitored environment that are receiving the monitored behavior (e.g., the target of the threat).

[0171] Figure 6 A simplified flowchart of a system and processing involving reverse DNS (rDNS) requests and rDNS responses, according to some embodiments, is depicted.

[0172] At 610, source 602 (e.g., the packet sender) may send a message (e.g., one or more packets) to destination 604. The packets may include data about source 602, such as the IP address of the source 602 that sent the packet. For example, the IP address of source 602 may be included in the packet header. After destination 604 receives a message or a portion thereof from source 602, destination 604 may perform an rDNS request to attempt to determine the domain name associated with the IP address of source 602.

[0173] Figure 612 illustrates an rDNS request sent to DNS resolver 608. DNS resolver 608 can determine the domain name associated with an IP address by performing a lookup. If DNS resolver 608 has information about which domain name is associated with the IP address, it can return the domain name in response to a request that includes the IP address. If there is no domain name associated with the IP address obtained by DNS resolver 608, the rDNS request may fail.

[0174] At 614, DNS resolver 608 can transmit a response to the rDNS request. The response may include the domain name of source 602 (e.g., a fully qualified domain name (FQDN)), which was obtained using the IP address associated with source 602. The response may be sent to target 604, which originally made the rDNS request.

[0175] The embodiments included herein may use a traffic monitoring system 606 to monitor rDNS requests sent from one or more targets 604 to one or more DNS resolvers 608. Furthermore, the traffic monitoring system 606 may be configured to monitor rDNS responses sent from one or more DNS resolvers 608 to one or more targets 604 as responses to rDNS requests. Traffic monitoring system 606 may be able to obtain data from rDNS requests and responses, such as timestamps (Ts), messages (e.g., source port 602, fully qualified domain name of source 602, response) (Msg), IP address of source 602, IP address of destination 604 (Src), query length of fully qualified domain name (qLen), query type (qType), response code (rc), time-to-live (ttl) value, value used to map the query back to the virtual network (VCNTSig), internal name server views used for resolution (e.g., viewHash) (e.g., namespaces used for internal and / or non-internet routing), markers used to track whether the resolution path is via the internet or local (path), and / or other data contained in packets sent between destination 604 and DNS resolver 608. The data that traffic monitoring system 606 can obtain may be referred to as raw data.

[0176] Examples of formats for the raw data and its included fields could be: {"ts":1620002030957,"msg":"34157 90.73.0.10.in-addr.arpa.: no-answer","src":"10.9.90.17","qLen":24,"qtype":"PTR","rc":3,"ttl":10,"vcnTsig":"jbrjswk3sjyudsynektrvq==.","viewHash":"qw5guonijczl6nzkpsvmhmnpfe.view.alt.","path":"p"} or {"ts":1620002030963,"msg":"43377 212.47.115.62.in-addr.arpa.: [PTR las-b24-link.ip.twelve99.net.]","src":"10.7.108.21","qLen":27,"qtype":"PTR","rc":0,"ttl":3021,"vcnTsig":"x9elmuu0rectetf4cmpizq==.","path":"i"}.

[0177] Traffic monitoring system 606 can be included in cloud defense system. Cloud defense system will be discussed in more detail below.

[0178] Figure 7 A simplified flowchart illustrating a system and processing for obtaining monitored traffic data from rDNS requests and rDNS responses, according to some embodiments, is depicted. System 700 illustrates how the system and methods described with respect to system 600 can be used and scaled in a cloud computing environment.

[0179] System 700 illustrates how multiple sources (e.g., source A 702a and source B 702b) each transmit one or more packets to one or more host machines (e.g., host machine A 706a) via a VCN (e.g., VCNA 704a). For example, a source (e.g., source A 702a) may transmit one or more packets to one or more host machines (e.g., host machine A 706a). Furthermore, one or more host machines (e.g., host machine A) may receive one or more packets from one or more sources (e.g., source A 702a).

[0180] Packets can be sent to a host machine via the corresponding host machine for various reasons, such as when an email is sent from source A 702a to host machine A 706a. In another example, a packet is sent to host machine B 706b when source B 702b is transmitting openSSH commands to host machine B 706b.

[0181] In some embodiments, a VCN comprises one or more host machines. In system 700, it is illustrated that a VCN may include one or more host machines. Specifically, the illustrated VCN A 704a includes host machine A 706a, host machine B 706b, and any other number of host machines, such as those represented by host machine N 706n. As further described herein, in some embodiments, one or more regions may include one or more VCNs.

[0182] After host machine A 706a receives one or more packets from source A 702a, host machine A 706a can perform an rDNS request via VCN A 704a using DNS resolver A 608a. As described with respect to system 600, system 700 is illustrated as being configured to obtain traffic data from the rDNS requests and responses between the host machine and the DNS resolver using cloud defense system 708 (which may include traffic monitoring system 606). Requests and responses sent from and to the host machine can be sent and received via the corresponding VCN. Therefore, the raw data discussed above with respect to system 600 can be obtained using cloud defense system 708. The monitored raw traffic data can be collected and stored as raw data 710.

[0183] Additionally, the cloud defense system 708 can monitor any number of rDNS requests made by host machine B 706b or any other host machine to any of the possible DNS resolvers. The cloud defense system 708 can also monitor the corresponding responses to the rDNS requests.

[0184] Because the cloud defense system 708 collects monitored traffic data from one or more rDNS requests and / or responses between one or more host machines and one or more DNS resolvers, raw data 710 can be collected and stored by the cloud defense system 708. Raw data 710 can be used by the cloud defense system 708 to generate enhanced data 712.

[0185] Figure 8 A simplified architecture diagram of a system that communicates with and constitutes the cloud defense system 708 according to some embodiments is shown.

[0186] System 800 illustrates a traffic monitoring system 606 configured to monitor rDNS request and / or response traffic between a cloud service provider infrastructure (CSPI) 802 and one or more DNS resolvers. Components may be logical components, physical components, or a combination thereof.

[0187] As illustrated in system 700, rDNS requests can be sent from a VCN to a DNS resolver. As already discussed, cloud defense system 708 can monitor rDNS requests and / or responses sent between one or more VCNs and one or more DNS resolvers via traffic monitoring system 606. System 800 illustrates that in some embodiments, the cloud service infrastructure may include one or more regions, and each region may include one or more VCNs.

[0188] Traffic monitoring system 606 can be configured to monitor traffic in an environment (e.g., a distributed environment). The monitored environment may include one or more regions. The monitored environment may include one or more VCNs. Furthermore, each region may include one or more VCNs. In some embodiments, the monitored environment is part of one or more regions. In some embodiments, multiple traffic monitoring systems 606 can be used with CSPI 802 to monitor different environments, the same environment, or different portions of CSPI 802. Therefore, each traffic monitoring system 606 of cloud defense system 708 can be configured to monitor at least a portion of CSPI 802. In some embodiments, a first traffic monitoring system 606 monitors an environment included in a first CSPI 802, and a second traffic monitoring system 606 monitors an environment included in a second CSPI 802.

[0189] As an example of how the traffic monitoring system 606 can be configured, the traffic monitoring system 606 can be configured to monitor rDNS requests and / or responses sent between VCN A 704a and one or more DNS resolvers. Therefore, the raw data 710 collected from the traffic monitoring system 606 can be able to include rDNS request and / or response data known to be associated with a specific VCN (e.g., VCN A 704a).

[0190] As another example, the traffic monitoring system 606 can be configured to monitor rDNS requests and / or responses sent from and / or to a region (e.g., from and / or to at least a portion of a VCN within the region). Therefore, the raw data 710 collected from the traffic monitoring system 606 can include rDNS request and / or response data known to be associated with a specific region (e.g., region A 804a) and at least a portion of its VCN.

[0191] In some embodiments, the traffic monitoring system 606 is configured to monitor rDNS requests and / or responses made at a global (e.g., all regions) level. Therefore, the raw data 710 collected from the traffic monitoring system 606 may be able to include rDNS request and / or response data known to be associated with a specific set of regions and / or VCNs (e.g., VCNs from region A 804a to region N 804n).

[0192] Additionally, in some embodiments, one or more DNS resolvers receiving one or more rDNS requests from one or more VCNs, regions, and / or CSPI 802, etc., may be located within the environment where the cloud infrastructure resides (e.g., DNS resolver C 708c). In some embodiments, one or more DNS resolvers are not local to the network in which CSPI 802 is operating (e.g., DNS resolver A 608a, DNS resolver B 608b, DNS resolver N708n). System 800 illustrates that any number of DNS resolvers (within or outside CSPI 802) may be capable of receiving rDNS requests from VCNs, where VCNs may be within one or more regions and / or CSPI 802.

[0193] As shown in systems 600 and 700, raw data 710 is obtained from monitoring rDNS requests and / or responses sent between CSPI 802 (e.g., VCN A704a, VCN B 704b, and / or Zone B 804b). Raw data 710 may be stored in memory repository 808. Data that may be included within raw data 710 has already been described above with respect to system 600.

[0194] Alternatively, or as an alternative to obtaining raw data 710 from traffic monitoring system 606, raw data 710 may be obtained from external data source 810. For example, external data source 810 may be a data source containing data related to: network activity, IP address owners, fully qualified domain name owners, known scanners, information about known attack methods (e.g., an attack that might cause a host to request a specific record to trigger a vulnerability in a recursive resolver within a VCN or CSPI, or to extract additional metadata about the environment), etc. External data source 810 may include data obtained from other traffic monitoring systems 606 and / or other cloud defense systems 708. In example attack activities that may be logged in external data source 810, an attacker defines a PTR for the IP performing a brute-force attack, which contains specific data or structures intended to enumerate detailed information about the DNS infrastructure using attempted connections.

[0195] External data source 810 can provide information to be used as raw data 710 or information to be combined with raw data 710. For example, raw data 710 may include a first fully qualified domain name associated with a first rDNS response, and external data source 810 may signal that the fully qualified domain name is a known scanner (e.g., a scanner that should not be blocked, a scanner that should be blocked), and therefore external data source 810 can be used to supplement the operation of data augmentation system 812. Thus, in some embodiments, external data source 810 is able to transmit data to data augmentation system 812 to assist data augmentation system 812 in creating augmented data 712.

[0196] The data augmentation system 812 may be able to generate augmented data 712 using the original data 710. In some embodiments, the data augmentation system 812 additionally or alternatively uses an external data source 810 to generate the augmented data 712 (e.g., a database of the augmented data 712 or a portion thereof, regional Internet registry information, etc.). In some embodiments, the augmented data 712 includes at least the original data 710.

[0197] Enhanced data 712 may include data from original data 710. For example, enhanced data 712 may include timestamps (Ts), messages (e.g., source port, fully qualified domain name of the source, response) (Msg), source IP address, target IP address (Src), query length of the fully qualified domain name (qLen), query type (qType), response code (rc), time-to-live (ttl) value, value used to map the query back to a virtual network (VCNTSig), internal name server view used for resolution (e.g., namespaces for internal and / or non-internet routing) (viewHash), a flag used to track whether the resolution path is via the internet or local (path), and / or other data contained in packets sent between the target and the DNS resolver.

[0198] Therefore, the information including enhanced data 712 can also be obtained using registry data (e.g., regional internet registry). Registry data can be used to identify the owner of the IP address, the IP address range in the NetRange, the organization, and / or other associated details. For example, a PTR or pointer record can provide a reverse mapping between an IP address and a domain name. A PTR can be the inverse of a PTR record, and the domain name mapped to by the PTR record can provide context about the interaction with the IP address or be used to associate the IP address with a larger infrastructure.

[0199] Furthermore, routing table data (e.g., RIPE, routeviews) can be used to identify who is routing the prefix associated with the IP address. In some embodiments, if the OriginAS value is defined for a prefix in Regional Internet Registry (RIR) data but does not match the routing data, a flag is associated with the prefix associated with the IP address. This flag can indicate data anomalies.

[0200] In some embodiments, the enhanced data 712 may include the reverse sender's IP address as the subject of the rDNS request, the fully qualified domain name (FQDN), the network identifier (e.g., TSIG), the zone, the rDNS response code, the network source from which the network traffic originates (e.g., Autonomous System Number (ASN) ("OriginAS")), the owner of the network from which the network traffic originates (e.g., the prefix owner ("Organization")), NetRange, CIDR, NetName, NetHandle, Parent, NetType, RegDate, Updated, Address, City, StateProv, Postal Code, Country, OrgAbuseHandle, OrgAbuseName, OrgAbusePhone, OrgAbuseEMail, OrgID, ASNumber, ASName, ASHandle, and / or other information associated with the IP address of the sender as the subject of the rDNS request.

[0201] Accordingly, each rDNS request can result in the generation of raw data 710, which can then be used to generate enhanced data 712. The data enhancement system 812 can then transfer the generated enhanced data 712 to a storage repository 808.

[0202] The augmented data 712 can be used to perform data analysis using the data analysis system 818. In some embodiments, the augmented data 712 may be clustered by the data augmentation system 812 to facilitate data analysis performed by the data analysis system 818. Therefore, the augmented data 712 may include clustered augmented data and / or non-clustered augmented data.

[0203] Clustering can be performed to group similar augmentation data together. For example, augmentation data is likely similar if the IP addresses (e.g., source IP, destination IP, IP looked up using rDNS requests) are the same, the TSig (each VCN can have a unique VCN TSig) is the same, the prefix is ​​the same, the autonomous system is the same, etc. Therefore, clustering enables the grouping of pointer-related observations. For example, clustering can group augmentation data by pointer namespace to assist in identifying clusters of infrastructure. It is possible to identify clusters of infrastructure by examining augmentation data to determine patterns of associated infrastructure that may originate from different prefixes or autonomous systems. In some embodiments, the false positive rate can be reduced by using a list of common suffixes.

[0204] As an example, IP addresses 209.141.58.151, 198.98.59.197, 209.141.35.27, and 199.195.254.209 can be mapped to the namespaces exit01.oxds().org, exit03.oxds().org, exit10.oxds().org, and exit17.oxds().org, respectively. Therefore, the augmented data associated with each of the IP addresses in the above example can be grouped into an associated cluster of namespaces, all of which are associated with the oxds().org namespace.

[0205] As a result of clustering, at least the total query count for each region, each CSPI 802, each VCN, each sender IP, each IP in the pointer, and / or each VCN network identifier (e.g., TSIG) can be determined. By grouping similar information together, the scale of the information can be used to determine trends (e.g., anomalies, patterns) in the augmentation data 712. By analyzing the augmentation data clusters, analysis of relevant information can be performed, providing more information than non-clustered augmentation data for individual rDNS requests and / or responses. The results of clustering can also provide an understanding of whether individual nodes are used to initiate network traffic or whether a superset of infrastructure is being used. Additionally, a larger number of nodes associated with namespaces can be identified by clustering the augmentation data.

[0206] Enhanced data 712 (clustered and / or non-clustered enhanced data) can be transmitted to alerting system 816. Alerting system 816 can use enhanced data 712 to determine whether an alert should be generated based on the received enhanced data 712. Alerting system 816 may be a pre-existing alerting system capable of generating alerts for one or more systems (e.g., a cloud defense system or other systems) using enhanced data 712 obtained from traffic monitoring system 606 and data enhancement system 812. Alerts may cause the generation of reports (e.g., logs, emails, printouts), configuration (e.g., setting flags), or other actions. In some embodiments, the alerting system may be able to communicate with the cloud defense system to generate and / or respond to the generation of alerts.

[0207] Augmented data 712 (e.g., clustered augmented data and / or non-clustered augmented data) can be transmitted to data analysis system 818. Data analysis system 818 can analyze augmented data 712 to identify trends, patterns, anomalies, generate reports, respond to queries, perform system configurations, determine the origin of network traffic, and determine observed and / or expected baseline network traffic, etc. Data analysis system 818 may include analyzer subsystem 820, report generation system 822, and / or query system 824.

[0208] The analyzer subsystem 820 can analyze the augmented data 712. The analyzer subsystem 820 can generate one or more alarms or reports. Furthermore, the analyzer subsystem 820 can identify one or more patterns in the augmented data 712. The identified patterns can represent baseline network activity or anomalous network activity.

[0209] For example, baseline network activity can represent one or more IP addresses, VCNs, zones, CSPI 802s, networks, fully qualified domain names, and / or IP address owners that are sending and / or receiving requests (e.g., within or outside CSPI 802). Baseline network activity can represent the average number of rDNS requests made by a particular VCN, zone, CSPI 802, etc., over a time period (e.g., seconds, minutes, hours, days, weeks, months, years, etc.), time of day, time of year, etc. Baseline network activity can represent the fully qualified domain names and / or IP addresses that are the subjects of rDNS requests within a time period. Baseline network activity can represent a baseline of any combination of measurements included in Enhancement Data 712.

[0210] Baseline network activity can be determined using an average (e.g., the average number of packets from the first IP address) or other measures that represent previous trends observed by the traffic monitoring system 606 and are related to any portion of the enhanced data 712 that can be obtained by using the traffic monitoring system 606.

[0211] Baseline network activity can be used to determine what kind of network traffic is expected in a specific context (e.g., time of day). Baseline network activity can be used to set a baseline for the number of rDNS resolver requests and / or responses sent and / or received by a monitored environment (e.g., VCN, zone). In some embodiments, baseline network activity can be used to determine how many rDNS requests include the same IP address within a given time period. In some embodiments, baseline network activity can be used to determine how many rDNS request responses include the same FQDN or are associated with the same owner within a given time period.

[0212] Baseline network activity can be used to compare with network activity subsequently monitored in the same or different monitored environments to help determine whether anomalous network activity is occurring, what environment or part of it is the subject of the anomalous network activity, and / or to help determine what is causing the anomalous network activity. Accordingly, baseline network activity can be used to set network activity thresholds so that any activity exceeding the network activity thresholds can be categorized and / or further analyzed.

[0213] Establishing a baseline network activity for the monitored environment by observing its network activity can reduce false positives for anomalous network activity. This is because observations can be compared to the baseline activity in various ways (comparing to the network activity of the monitored environment, the network activity of a subset of the monitored environment, and considering other contextual information available within the data to enhance its impact). Therefore, obtaining baseline network activity can be used to more accurately identify what constitutes anomalous network activity.

[0214] For example, in some embodiments, a monitored environment may be considered under attack when a certain number of rDNS requests are generated by the monitored environment (e.g., VCN, region) in a first time period, and this number is significantly greater than (e.g., 50% greater) the baseline rDNS requests in a second time period. The first and second time periods may be the same amount of time (e.g., one minute) and / or the same time window (e.g., from noon to midnight on the first day compared to noon to midnight on the second day).

[0215] In some embodiments, the current network activity of the monitored environment can be compared with a corresponding baseline network activity metric for the same monitored environment or different monitored environments (e.g., VCN, region, and / or other monitored environments).

[0216] In some embodiments, any number of actions (e.g., zero or more) can be performed when the network activity of the monitored environment differs sufficiently from the baseline network activity.

[0217] Actions that can be performed in response to the detection of network activity sufficiently different from baseline network activity may include determining at least one of the following: the portion of the monitored environment targeted by the network activity (e.g., area A 804a, VCN A704a) and the source(s) generating the network activity (e.g., one or more IP addresses, and the owners of one or more IP addresses). In some embodiments, additional information related to the anomalous network activity, such as the port associated with the network activity, may also be collected.

[0218] The data analysis system 818 may also include a report generation system 822. The report generation system 822 may be able to generate reports based on at least one of the following: analysis performed by the analyzer subsystem 820, enhanced data 712, and input from the query system 824.

[0219] The report generation system 822 can generate reports that include specific information. The information included in the generated report may depend on the type of report already generated, the reason for generating the report, the parameters used when initiating report generation, and / or the data included in the report.

[0220] As an example, when the analyzer subsystem 820 compiles network activity of the monitored environment (e.g., baseline activity, anomalous activity, etc.), such as at least a portion of the enhancement data 712 (e.g., data included in rDNS requests and responses), the analyzer subsystem 820 can cause the report generation system 822 to generate a report.

[0221] When a user of query system 824 submits a request for query system 824 to obtain or generate a report, query system 824 may cause report generation system 822 to generate the report. Users of query system 824 may have the ability to specify what data they wish to include in the generated report. In some embodiments, users may be limited (e.g., by user permissions) to the data they can request from the query generation system, thereby limiting the data that may be included in the generated report.

[0222] The report generation system 822 may also monitor augmentation data 712 to determine whether a report should be generated. In some embodiments, the report is generated when a condition occurs (e.g., an unusual event occurs, a request is submitted), or it may be generated on a schedule (e.g., a report is generated once a day).

[0223] The report generation system 822 can transmit the generated report. The generated report can be transmitted to another system (e.g., user equipment, printer, another system, etc.).

[0224] The data analysis system 818 may also include a query system 824. The query system 824 may be able to receive queries generated by other systems and / or users. Queries received from the query system 824 may be received from users and / or systems within the cloud defense system 708 environment (e.g., the administrator of the cloud defense system 708, user B 826) and / or from users and / or systems outside the cloud defense system 708 (e.g., public users, customers of cloud service providers, other systems capable of using enhanced data 712, user A 828, etc.). The query system 824 may be able to receive query input and perform searches on the enhanced data 712 to generate output in response to the received query.

[0225] In some embodiments, the query system 824 is configured to transmit information to the system or user based on whether the system or user is within the cloud defense system 708 or has a permission level determined in another way.

[0226] In some embodiments, other systems may be configured to use query system 824 to obtain enhanced data 712 from cloud defense system 708, enabling these systems to perform further actions. In some embodiments, alert system 816 may be configured to use query system 824.

[0227] Figure 9 A simplified flowchart 900 depicts a method for protecting a monitored environment using rDNS traffic, according to some embodiments. Figure 9 In the example embodiment depicted, the process described in flowchart 900 can be executed by cloud defense system 708.

[0228] At 902, information identifying the environment to be monitored (e.g., a distributed environment) is received. The monitored environment may include a portion of infrastructure provided by the CSP to provide one or more cloud services to the CSP's customers. The monitored cloud environment may include one or more VCNs running payloads for the CSP's customers. For example, the monitored environment may be a CSP's data center, a portion of a data center, multiple data centers in a region, infrastructure in multiple regions, global infrastructure, etc. In some embodiments, the information received at 902 may identify one or more VCNs to be monitored (e.g., a unique network identifier (VCN TSig) for the VCN), such as one or more VCNs belonging to one or more customers (the one or more VCNs may be associated with one or more customers, regions, etc.).

[0229] At position 904, within a specified time period, rDNS traffic associated with the monitored environment identified in position 902 is monitored. Monitoring rDNS traffic at position 904 may include monitoring requests originating from the monitored environment and / or monitoring responses to rDNS requests received from one or more DNS resolvers for the monitored environment. (See above regarding...) Figure 6 As described in process 600, an rDNS request typically results in a corresponding rDNS response being received from the DNS resolver. However, in some cases, the DNS resolver may not generate a response. This can happen, for example, when the DNS resolver does not store a pointer record for the IP address identified in the rDNS request.

[0230] At 906, based on the monitoring performed in 904, data related to the monitored rDNS traffic is collected and stored for the monitored environment. For the purposes of this disclosure, the data collected and stored at 906 may be referred to as raw data. This is to distinguish this data from the enhanced data discussed below. The term raw data is not intended to limit the scope of the claimed embodiments in any way. Data that may be included in rDNS requests and / or responses and that may be collected and stored in 906 has already been discussed above. Figure 6 The process 600 in the document is described.

[0231] Raw data can be stored in various formats, such as documents, tables, files, data repositories, data lakes, databases, etc. Raw data can be stored locally and / or remotely within the cloud defense system.

[0232] At point 908, the raw data collected and stored in point 906 is enhanced with additional information to generate enhanced data. There are various methods that can enhance the raw data. For example, enhancement may include at least one of the following: adding or supplementing the raw data with additional information, organizing the raw data in a desired manner to facilitate analysis (e.g., clustering the data along one or more dimensions), or replacing at least a portion of the raw data.

[0233] The raw data may include: timestamp (Ts), message (e.g., source 602 port, fully qualified domain name of source 602, response) (Msg), IP address of source 602, IP address of destination 604 (Src), query length of fully qualified domain name (qLen), query type (qType), response code (rc), time to live (ttl) value, value used to map the query back to the virtual network (VCNTSig), internal name server view being used for resolution (e.g., namespaces used for internal and / or non-internet routing) (e.g., viewHash), flag used to track whether the resolution path is via the internet or local (path), and / or other information contained in packets sent between destination 604 and DNS resolver 608.

[0234] Information added to the raw data can be obtained from various sources, including sources within and outside the monitored environment. Sources outside the monitored environment can include those provided by the CSP or by other third parties. For example, additional raw data (e.g., collected by another monitored environment at different times) can be added to the raw data to generate augmented data or a portion thereof. In another example, augmented data (e.g., collected by another monitored environment at different times by the same monitored environment) can be added to the raw data.

[0235] Supplementing the original data may include using the original data to obtain further data to be included in the enhancement. As an example, enhancement data may include the original data or portions of the original data. Enhancement data may include data obtained using registry data (e.g., a regional internet registry) to supplement the original data. Registry data may be able to identify the owner of the IP address, the IP address range within a NetRange, the organization, and / or other associated details. For example, a PTR or pointer record from the original data may provide a reverse mapping between IP addresses and domain names. A PTR may be the inverse of a PTR record, and the domain name mapped to by the PTR record may be able to provide context about interacting with the IP address or be used to associate the IP address with a larger infrastructure.

[0236] Furthermore, when supplementing raw data to generate enhanced data, routing table (e.g., RIPE, routeviews) data can be used to identify which prefixes associated with IP addresses are included in the original data in the routing. In some embodiments, if the OriginAS value is defined as a prefix in Regional Internet Registry (RIR) data but does not match the routing data, a flag is associated with the prefix associated with the IP address. This flag can indicate data anomalies.

[0237] Enhanced data can be generated by replacing at least a portion of the original data. Data fields included in the original data can be replaced with known, associated information. For example, the original data may include a first FQDN address, and the cloud defense system can be configured to replace the ".com" portion of the FQDN with a value representing the characteristics of the top-level domain (TLD) (e.g., ".com"). Such value replacement can reduce processing time. Furthermore, the values ​​can be replaced with encrypted values ​​to protect the privacy of one or more values ​​in the enhanced data.

[0238] Augmentation data can be generated by removing at least a portion of the original data. For example, time-to-live (TSI) values ​​can be removed because they may be deemed not to provide valuable insights into the monitored environment. Removing such information can reduce storage usage and processing time for augmentation data, allowing for faster identification of anomalous activity and quicker action. In some embodiments, removing at least a portion of the augmentation data can be performed to enhance privacy. For example, a customer using a monitored environment might want to prevent monitored network activity of one of their VCNs from being used to create augmentation data, or might want to remove VCN TSigs from the original data to create augmentation data.

[0239] Augmented data can be generated by organizing at least a portion of the original data. Organizing the original data can be performed to improve the efficiency of further analysis. For example, the original data can be organized at a hierarchical level. For instance, augmented data can be generated by organizing all rDNS requests from a first VCN into a single augmented data entry. In another example, augmented data can be generated by organizing all rDNS requests from a first region (e.g., the first VCN and the second VCN) into a single augmented data entry. In yet another example, augmented data can be generated by organizing all rDNS request responses that include the same FQDN into a single augmented data entry. Accordingly, the organization of the original data and / or augmented data can be performed across multiple dimensions and can be referred to as "clustering".

[0240] In some embodiments, the enhanced data may include the IP address of the reverse sender as the subject of the rDNS request, the fully qualified domain name (FQDN), the network identifier (e.g., VCN TSIG), the zone, the rDNS response code, the network source from which the network traffic originates (e.g., Autonomous System Number (ASN) ("OriginAS")), the owner of the network from which the network traffic originates (e.g., prefix owner ("Organization")), NetRange, CIDR, NetName, NetHandle, Parent, NetType, RegDate, Updated, Address, City, StateProv, Postal Code, Country, OrgAbuseHandle, OrgAbuseName, OrgAbusePhone, OrgAbuseEMail, OrgID, ASNumber, ASName, ASHandle, and / or other information associated with the IP address of the sender as the subject of the rDNS request.

[0241] Enhanced data can be used to determine baseline network activity for a monitored environment or a portion thereof. In a first example, enhanced data can be used to determine baseline rDNS requests sent by the monitored environment or a portion thereof (e.g., VCN). In a second example, enhanced data can be used to determine whether the monitored environment observes activity different from the baseline network activity for the monitored environment. The baseline network activity for the monitored environment can be based on previous enhanced data obtained from the same or different monitored environments, can be configured by the user of the cloud defense system, and / or can be based on one or more external data sources.

[0242] At 910, the enhanced data is used to identify anomalous network activity associated with the monitored environment. In some embodiments, the processing performed at 909 may include the processing performed at 912 and 914.

[0243] At point 912, the augmented data generated in point 908 is used to identify components of the monitored environment that may be potential targets of malicious activity. The augmented data can be used to determine if the monitored environment is observing behavior that differs from baseline network activity. If a difference threshold is exceeded, the monitored network activity may differ from the baseline network activity of the monitored environment. The difference threshold can be set using configuration made by the user of the cloud defense system and / or the monitored baseline data.

[0244] Some examples of thresholds that can be evaluated are: the number of rDNS requests made by a specific monitored environment (e.g., VCN, region) within a given time period; the number of rDNS requests including the first piece of information (e.g., IP address); the number of rDNS request responses including the second piece of information (e.g., fully qualified domain name); the number of rDNS requests within clusters of augmented data; the number of rDNS requests identified as being associated with an IP address owner; the time between rDNS requests including one or more pieces of identical information; and so on. Thresholds may relate to one or more fields of information from the augmented data. In some embodiments, machine learning models may be used to determine whether a baseline is sufficiently different from the augmented data (e.g., to determine whether the similarity between the baseline and the augmented data is within a threshold).

[0245] Another example of a threshold that can be evaluated is the number of rDNS requests that do not return a response, and as a two-phase analysis, this includes the number of FQDNs returned and the number of times that FQDN is correctly resolved with NXDomain / ServFail. Another example of a threshold that can be evaluated is the number of rDNS requests for paired IPv4 and IPv6 addresses for dual-stack hosts, which can be used to establish or anchor links between v4 and v6 prefixes. Thresholds can be evaluated for public infrastructure, such as the provider of authoritative DNS for portions of the in-addr.arpa zone.

[0246] Therefore, the enhanced data obtained from the monitored environment can be compared with the monitored baseline network activity and / or user-selected baseline network activity to determine whether one or more parts of the monitored environment (e.g., VCN, region) are potentially the target of malicious attacks and / or are experiencing network activity that may be considered undesirable (e.g., based on user-defined expectations and / or past network activity), thereby indicating potential malicious activity.

[0247] At point 914, augmented data can be used to identify sources of malicious activity targeting the monitored environment. Sources can be identified by determining which sources (e.g., one or more IP addresses, one or more fully qualified domain names, one or more owners) correspond to augmented data that has exceeded a defined baseline threshold.

[0248] At point 916, a signal indicating anomalous network activity identified during step 910 can be output. This signal can be output to a component of the cloud defense system and / or to another system. A system external to the cloud defense system could be an alarm system for generating and transmitting alarms based on output signals from one or more systems. The signal can be generated and output during step 916 regardless of whether anomalous network activity is identified during step 910. Therefore, the signal can indicate whether anomalous network activity has occurred or not. Additionally, the process illustrated in flowchart 900 can be repeated an arbitrary number of times regardless of whether anomalous network activity is identified during step 910. For example, the process can be continuously executed by the cloud defense system to continuously monitor for anomalous network activity in the monitored environment. Therefore, the additional raw data collected can be augmented and compiled with existing augmented data for further analysis.

[0249] At point 918, one or more actions can be initiated in response to signals output during point 916. After one or more components of the monitored environment are identified as potentially vulnerable to attack and / or one or more sources of potential malicious activity are identified at point 914, any number of actions can be performed (e.g., by determining that abnormal network activity is occurring compared to one or more baseline network activity thresholds). For example, system configuration can be performed (e.g., taking a VCN offline, blocking IP addresses, dropping certain packets, setting up a firewall, generating logs, generating reports, patching systems, rerouting network traffic, contacting the source).

[0250] In examples of rerouting network traffic, in some embodiments, network traffic can be rerouted to a high-interaction honeypot. Available data allows the honeypot to match the specifications and services of the deployed host. The honeypot can allow connections to be established and observe the actions of the actor after it gains access to the host.

[0251] Figure 10 A simplified flowchart, according to some embodiments, is depicted for monitoring a monitored environment to determine baseline network activity and determining actions to be taken if anomalous activity is identified. Figure 10 In the example embodiment depicted in flowchart 1000, the process depicted in flowchart 1000 can be executed by cloud defense system 708.

[0252] At point 1002, the augmentation data already collected (e.g., within a time period) can be used to generate a baseline for the monitored environment. The augmentation data can be collected in a manner similar to obtaining augmentation data from the same monitored environment or different environments during periods 904, 906, and 908. In some embodiments, an algorithm or user-specified values ​​are used to generate the baseline network activity.

[0253] Enhanced data can represent a baseline for the monitored environment or a portion thereof. As an example, enhanced data could represent that the monitored environment transmits an average of 400 rDNS requests to resolve a first IP address within a first time period. Therefore, the baseline rDNS requests for that IP address, the corresponding FQDN identified in the possible rDNS request responses, the owner associated with that IP address, and / or the monitored environment can be determined to be 400 within a time period equal to the length of the first time period.

[0254] A baseline can be generated for the IP address that generated the rDNS request, the FQDN associated with the IP address, the owner associated with the IP address, the type of request, and / or any other information included in the enhancement data. A baseline can also be generated to determine the number of rDNS requests sent by the monitored environment or a portion thereof. Additionally, the baseline can be specific to one or more VCNs, a region, a client, the entire monitored environment, etc.

[0255] A baseline can have a range that is a portion (or all) of the monitored environment to which the baseline applies (e.g., the baseline represents network activity as a total for VCN A and VCN B, and the baseline represents network activity as an average for VCN A and VCN B). A baseline can also have a target, which is a component of the monitored environment for which the baseline is evaluated (e.g., determining whether VCN A (the target) exceeds a baseline threshold). Targets can include the same or fewer system components as the monitored environment.

[0256] Therefore, depending on the target, deviation from the baseline can be assessed for a single VCN or more VCNs within the monitored environment. As a further example of a monitored environment comprising a group of VCNs, a first baseline can be assessed for a group of VCNs (targets) within the range of the baseline (the VCN group) to make a single determination about whether there is a deviation from the baseline. In another example, a second baseline can be assessed for each VCN (target) within the range (the VCN group). Furthermore, each target within the monitored environment to be compared with the baseline can have its own (e.g., unique, independently determined) baseline threshold compared to other targets within the monitored environment.

[0257] At point 1004, in a manner similar to establishing a baseline by monitoring the monitored environment or a portion thereof, additional monitoring can be performed to determine if deviations from the baseline have occurred. Accordingly, at point 1004, augmented data collected for the monitored environment is used to identify deviations from the baseline.

[0258] Deviation from the baseline can be determined based on the fact that the baseline value for one or more augmented data information fields differs from the baseline value for one or more augmented data information fields (e.g., lower, higher, exceeding a threshold distance, etc.).

[0259] As an example, if the baseline number of rDNS requests sent by the monitored environment is 500 within one minute, the baseline might have been set to 500 plus or minus 50 (or 10%). Correspondingly, if the monitored environment sends more than 550 rDNS requests or fewer than 450 rDNS requests, an deviation could be flagged. Similarly, the baseline can have different levels of specificity, such as for how many rDNS request responses were resolved by the monitored environment, how many rDNS request responses were resolved to a specific FQDN, how far apart in time all rDNS requests related to resolving the same IP address or IP addresses owned by the same person are, or comparisons of other enhanced data fields.

[0260] At point 1006, if a deviation is identified during period 1004, that deviation can be flagged as an anomalous activity associated with the monitored environment. In response to whether the activity is flagged, one or more actions can then occur.

[0261] At point 1008, one or more actions can be performed in response to the identification of the tagged activity at point 1006. An alarm can be one of the actions performed.

[0262] At point 1008, a set of actions (e.g., alarms) can be identified. Each action in this set can have one or more conditions associated with it. Conditions can be related to information included within the augmented data. Thus, when a specific value or combination of values ​​exists within the augmented data, conditional logic may be satisfied, thereby triggering one or more actions. Additionally or alternatively, actions (e.g., alarms) can be time-based (e.g., reporting data at specific time intervals).

[0263] Actions may be related to whether a specific IP address was observed and / or whether a specific number of requests were observed from a single IP address, FQDN, or owner. Furthermore, actions may be related to whether an attack might be underway, whether micro-scanning was detected, and / or whether a scan was detected.

[0264] In some implementations, DNS service updates can occur as an action. For example, a two-phase PTR can return an FQDN (e.g., valid or invalid). If the FQDN is invalid and the IP belongs to a company, this could indicate a DNS inconsistency, causing the DNS service to be updated to remove the PTR, which is designated as a nominal maintenance activity. Actions can also send abuse reports to other providers. As another example of an action, observing multiple IPs from the same prefix, ASN, or associated namespace could be a sign of a larger problem, prompting alerts to other systems (e.g., within and outside the network) and potentially other networks.

[0265] Actions can be configured by users of the cloud defense system (e.g., internal user B, external user A), users of another system, processes of another system, etc.

[0266] For each action in this set of actions, the conditions for the action can be evaluated to determine whether the conditions for the action have been met and therefore the action should be executed. This determination can be made using the augmented data obtained at 1004. The determination can assess whether one or more conditions associated with the action are met. The determination can assess whether there are any flagged deviations at 1006. A non-exhaustive example of such a condition could be whether the corresponding VCN from the monitored environment is sending a number of rDNS requests that is an order of magnitude higher (e.g., 2x, 1.5x) or lower than the baseline for the VCN and / or the monitored environment. Another condition could be whether the corresponding VCN from the monitored environment is sending a number of rDNS requests that is a defined value higher (e.g., 100 requests) or lower than the baseline for the VCN and / or the monitored environment. Another condition could be whether the corresponding VCN from the monitored environment is sending a sufficiently high number of lookup requests for a specific IP address (e.g., a baseline for a specific IP address), a sufficiently high number of lookup requests for a specific IP address from the specific VCN making the request, and / or a sufficiently high number of lookup requests for a specific IP address from the specific monitored environment making the request. These conditions can be the same as or different from those assessed when determining whether anomalous network activity has occurred.

[0267] Furthermore, depending on the conditions that are met, different types of actions can be taken for the same identified anomaly (e.g., based on the severity of the anomaly (e.g., the difference from the baseline, the part of the monitored environment experiencing the anomaly, etc.)).

[0268] If the action condition is not yet met, the evaluation of the action condition can continue. For example, more augmented data can be collected to subsequently determine whether the action condition has been met (e.g., return to 1004).

[0269] If the action conditions are met, the corresponding action can be executed.

[0270] One or more actions can be initiated because one or more conditions for an action have been met. A non-exhaustive list of actions has been discussed above. Some example actions could be configuring a firewall, dropping packets, isolating a machine or part of a network, generating a report, transmitting a report, logging data, blocking an IP address, FQDN, and / or owner, or unblocking an IP address, FQDN, or owner (e.g., the IP address has not been seen for a certain period of time, or the number of packets received from the IP address has dropped (e.g., below the share or threshold of network activity)).

[0271] Figure 11 A simplified flowchart illustrating, according to some embodiments, for determining the baseline rDNS request volume for a monitored environment is depicted. Figure 11 In the example embodiment depicted, the processes described in flowchart 1100 may be executed by cloud defense system 708. Flowchart 1100 may be an example of the processes that occur during 1002.

[0272] At 1102, a group of one or more VCNs are identified as the source of one or more rDNS requests. These VCNs may also be within the monitored environment. The source of the rDNS request can be identified by determining the IP address used in the sender block of the rDNS request, or it can be identified using a unique network identifier (e.g., VCN TSig). The source of the rDNS request can be included in the enhancement data. The monitored environment can be of any scope and therefore can include one or more VCNs, zones, clients, etc.

[0273] At 1104, a determination is made for each VCN identified at 1102, which evaluates the rDNS information associated with the VCN from the enhancement data. The VCN-associated information may include any amount of data that can be obtained from rDNS requests and / or responses. For example, the amount of rDNS requests and / or responses to rDNS requests may be determined from the enhancement data within the scope of the monitored system, (one or more) zones and / or (one or more) VCNs, etc. In the example, additionally or alternatively, the amount of rDNS requests and / or responses to rDNS requests may be determined from the enhancement data relative to a specific set of one or more IP addresses, FQDNs, and / or owners. In the example, the number of rDNS requests that did not return an FQDN can be determined. In yet another example, one or more fields of the enhancement data may be determined using the rDNS information as described herein (e.g., above).

[0274] As a first example of determining the volume of rDNS requests, the monitored environment may include VCN A, VCN B, and VCN C. During the first time period, 1104 can determine that 400 rDNS requests originated from VCN A, 100 rDNS requests originated from VCN B, and 100 rDNS requests originated from VCN C.

[0275] At 1106, baseline network activity for the monitored environment can be determined based on the processing performed during 1104. Baselines can be generated for the entire monitored environment (e.g., one or more VCNs, one or more regions, one or more customers, etc.) or a portion thereof (e.g., one VCN, two VCNs, one region, one customer, etc.).

[0276] The volume of rDNS requests originating from a VCN, the IP addresses identified in these requests, and / or other information associated with the IP addresses identified in the requests during 1104 errors can be aggregated with one or more other VCNs to determine a baseline for the entire environment or a portion thereof. Baseline network activity may include time periods and / or augmented data collected within those time periods. Baseline network activity may be a statistical representation of rDNS requests originating from the monitored environment. This statistical representation may represent the entire monitored environment or a specific subset of rDNS requests originating from the monitored environment.

[0277] Using the first example above, at 1104, in some embodiments, baseline network activity for the monitored environment can be determined as 600 rDNS requests. In some embodiments, the baseline network activity of the first example can be more granular and can be determined to be an average of 400 rDNS requests made by VCN A, an average of 100 rDNS requests made by VCN B, and an average of 100 rDNS requests made by VCN C. Those skilled in the art who benefit from this disclosure will recognize that other methods can be used to determine a statistical baseline for the monitored environment from the obtained augmented data, not just using rDNS request counts, but using any combination of augmented data fields to form a particular baseline.

[0278] As discussed above and in more detail below, baseline network activity against a monitored environment or a portion thereof can be used to identify one or more components of the monitored environment that are experiencing unwanted network activity (e.g., potentially the target of malicious activity) and / or to identify the source of unwanted network activity (e.g., malicious activity) against at least a portion of the monitored environment.

[0279] The following is an example of how baseline network activity can be obtained for a range of VCNs, but similar techniques can be performed for other ranges. For each VCN identified at 1102, baseline network activity for each VCN can be determined. In some embodiments, baseline network activity is determined for a specific VCN.

[0280] Using the enhanced data, determine the volume of rDNS requests originating from a single VCN and the IP addresses identified in these requests for lookups (e.g., IP addresses used to look up fully qualified domain names). The volume of rDNS requests originating from a single VCN can be determined by summing the total number of rDNS requests generated by that single VCN. This summation of rDNS requests can be done on various basises, such as summing all rDNS requests, summing all rDNS requests that identified a unique IP address for the lookup, summing all rDNS requests that identified a matching IP address for the lookup, summing all rDNS requests based on additional information included in the enhanced data, etc.

[0281] As a first example, during the first time period, 400 rDNS requests originated from VCN A to look up IP address A, 100 rDNS requests originated from VCN A to look up IP address B, and 100 rDNS requests originated from VCN A to look up IP address C.

[0282] Baseline network activity can be determined for VCN A within a first time period. Baseline network activity can be a statistical representation of rDNS requests originating from VCN A. This statistical representation can represent the entire VCN A, or it can represent one or more specific subsets of rDNS requests originating from VCN A.

[0283] Continuing with the first example above, in some embodiments, the baseline for VCN A can be determined as 600 rDNS requests. In some embodiments, the baseline can be determined as VCN A making an average of 200 rDNS requests for any given lookup of an IP address. In some embodiments, the baseline of the first example can be more refined and can be determined to have an average of 400 rDNS requests for IP address A, an average of 100 rDNS requests for IP address B, and an average of 100 rDNS requests for IP address C. Those skilled in the art who benefit from this disclosure will recognize other ways in which the obtained enhanced data can be used to determine statistical baselines on a per-VCN basis.

[0284] Similar baseline representations can be determined for any range of monitored environments (e.g., multiple VCNs, regions, multiple areas, etc.).

[0285] Figure 12 A simplified flowchart for detecting anomalous network behavior in a monitored environment, according to some embodiments, is depicted. Figure 12 In the example embodiment depicted, the processes described in flowchart 1200 can be executed by cloud defense system 708. Flowchart 1200 is an example of a baseline having a range including one or more VCNs and targets of VCNs within the monitored environment. However, embodiments that detect anomalous activity across different ranges within the monitored environment are also contemplated, such as monitored environments including more than one region, more than one customer, etc.

[0286] At position 1202, augmentation data for the monitored environment within the specified time period is obtained. This augmentation data can be obtained in a manner similar to, for example, the methods described above for obtaining augmentation data at positions 902, 904, 906, and 908. For instance, rDNS requests originating from VCNs within the monitored environment can be used to generate augmentation data.

[0287] At 1204, for each corresponding VCN (target) within the monitored environment (range), steps 1206, 1208, 1210, 1212, and 1214 can be performed. These steps compare augmented data from the target within the monitored environment with baseline network activity for the monitored environment (represented by baseline augmented data for the monitored environment) to determine whether one or more actions should be performed. The steps performed can be examples of comparing augmented data from a specific target VCN within the range of the monitored environment (which includes one or more VCNs, e.g., a region) with baselines (e.g., obtained from the monitored environment, a portion of the monitored environment, or another monitored environment). For each target, any number of baselines can be evaluated, and any number of targets within the range, equal to or less than the system component count, can be evaluated for one or more anomalies.

[0288] At 1206, using the augmented data collected during 1202 by monitoring the environment, the number of rDNS requests originating from the corresponding VCN within a time period can be determined. This time period can be the same time period observed when baseline augmented data of baseline network activity was obtained (e.g., the same amount of time, the same time of day) or a similar time period (e.g., 1.2 hours compared to 1 hour).

[0289] At 1208, the information determined during 1206 can be compared with the baseline for the monitored environment. Any (one or more) portions of the augmented data related to the corresponding VCN of the monitored environment acquired during 1202 can then be compared with the baseline for the monitored environment, a portion thereof (e.g., its VCN, the same corresponding VCN).

[0290] At 1210, a determination is made regarding the presence of anomalies based on the comparison performed during 1208 (comparing the monitored environment with baseline network activity for the monitored environment).

[0291] An anomaly can be identified if the baseline differs sufficiently from augmented data obtained from at least a portion of the monitored environment. For example, the average rDNS request count of at least a portion of the monitored environment (e.g., a specific VCN) can be compared to at least a portion of the baseline (e.g., the baseline average VCN rDNS request count for the environment or the baseline for a specific VCN). Figure 11 Any data collected during the processing of baseline network activity augmentation data can be used for comparison with augmentation data information related to the monitored environment identified at 1206 locations.

[0292] In addition, at least one of a variety of measures can be used to determine that the baseline network activity is sufficiently different from the monitored environment. One measure could be whether the number of rDNS requests being sent by the corresponding VCN from the monitored environment is one order of magnitude higher (e.g., 2 times higher, 1.5 times higher) or lower than the baseline for the VCN and / or the monitored environment. Another measure could be whether the corresponding VCN from the monitored environment is sending a number of rDNS requests that are higher than a defined value (e.g., 100 requests higher) or lower than the baseline for the VCN and / or the monitored environment. Another measure could be whether the corresponding VCN from the monitored environment is sending a sufficiently higher number of rDNS requests to look up a specific IP address than the baseline (e.g., the baseline for a specific IP address), a sufficiently higher number of rDNS requests to look up a specific IP address than the baseline of the specific VCN making the request, and / or a sufficiently higher number of rDNS requests to look up a specific IP address than the baseline of the specific monitored environment making the request. Another measure could be to compare the standard deviation of the obtained augmentation data with the baseline of the corresponding value for the monitored environment or a portion thereof.

[0293] Various other conditions and combinations of conditions can be used to compare one or more values ​​from augmented data with a baseline to determine which activities should be defined as anomalous and to help provide insights into the activities taking place within the monitored environment.

[0294] Such conditions can enable the identification of anomalous behavior compared to a baseline. These include, but are not limited to: scans from new sources, a reduction in scans from existing sources, sources of denial-of-service attacks, and / or insider threats.

[0295] At point 1212, if no anomaly is detected during period 1210, monitoring of the monitored environment can continue by returning to point 1202. If an anomaly is detected during period 1210, one or more actions can be initiated in response during period 1214.

[0296] If an anomaly is detected in the monitored environment during step 1210, step 1214 can be executed. In some embodiments, actions such as reducing restrictions on system access to the network can still be performed if no anomaly is detected. If an anomaly is detected, a signal identifying the anomaly can be output. The signal can be an email, text message, or other transmission.

[0297] As a result of the output signal in 1214, any number of actions can be performed during 1216. In some embodiments, system configuration may occur, such as configuring a router to drop packets, configuring a firewall, configuring a VCN to drop packets, configuring a VCN to be added to a block list, generating reports, logging at least a portion of the enhanced data from the monitored environment, transmitting communications, disconnecting the VCN from the network, or other similar actions, to prevent a known attacker from sending packets to the monitored environment or the network associated with the monitored environment.

[0298] Similar processing to that described in flowchart 1200 can also be applied to different targets and / or ranges in the monitored environment, thus allowing fine-grained control over the baseline, appropriate baseline threshold deviation, targets, ranges, and actions.

[0299] Figures 13A-13B A simplified flowchart for detecting malicious actors using a cloud defense system, according to some embodiments, is depicted. The simplified flowchart can be used to identify actors acting in an undesirable manner (e.g., maliciously) relative to a VCN, region, CSPI, or multiple CSPIs. In the example embodiment depicted in FIG13, the processes depicted in flowchart 1300 can be performed by cloud defense system 708.

[0300] At 1302, an IP address level threshold for the rDNS request is determined. The IP address level threshold can be based on a predetermined IP address level threshold (e.g., user-specified) and / or a baseline value for the rDNS request. For example, the IP address level threshold could be 500 rDNS requests associated with a single IP address in the environment. The first IP address level threshold for the first IP address can be the same as or different from the second IP address level threshold for the second IP address.

[0301] At 1304, an FQDN level threshold associated with the rDNS request is determined. The FQDN level threshold can be based on a predetermined FQDN level threshold (e.g., user-specified) and / or a baseline FQDN level threshold for the rDNS request. For example, the FQDN level threshold could be 500 rDNS requests associated with a first FQDN in the environment. The first FQDN level threshold for the first FQDN can be the same as or different from the second FQDN level threshold for the second FQDN.

[0302] At 1306, an owner-level threshold associated with the rDNS request is determined. The owner threshold can be based on a predetermined owner threshold (e.g., user-specified) and / or a baseline owner threshold for the rDNS request. For example, the owner threshold could be 500 rDNS requests associated with the first owner of the environment. The first owner-level threshold for the first owner can be the same as or different from the second owner-level threshold for the second owner.

[0303] At point 1308, enhanced data is obtained for the monitored environment. Enhanced data can be obtained by monitoring the monitored environment over a time period. The monitored data can reflect network traffic over the time period (e.g., the monitored environment, a portion of the monitored environment, or different environments). Enhanced data can also be obtained using data obtained from rDNS requests and / or responses to rDNS requests.

[0304] At 1310, using enhanced data, a set of one or more unique IP addresses involved in one or more rDNS requests can be identified. In some embodiments, the identified set of one or more IP addresses is a subset of the IP addresses involved in all rDNS requests. For example, this subset can be determined based on TSIG, VCN, zone, etc. This may be because in some embodiments, it may not be desirable to analyze certain rDNS requests (e.g., additional privacy is desired). The identified IP addresses can then be used to cluster the enhanced data associated with each IP address (e.g., to determine how many rDNS requests are associated with each IP address).

[0305] At 1312, using augmented data (e.g., the group of one or more IP addresses), a group of one or more unique fully qualified domain names (FQDNs) can be identified that are involved in one or more rDNS requests (e.g., as indicated by the rDNS request response). Each FQDN can be returned from a DNS resolver in response to a DNS request that includes the IP address associated with that FQDN. The identified FQDNs can then be used to cluster the augmented data associated with each FQDN (e.g., to determine how many rDNS requests are associated with each FQDN).

[0306] At point 1314, using augmented data (e.g., one or more IP addresses in the group, one or more unique FQDNs in the group), one or more owners involved in one or more rDNS requests can be identified (e.g., using whois IP lookups, using an obtained IP owner database, using an obtained FQDN owner database). The identified one or more owners can then be used to cluster the augmented data associated with each owner (e.g., to determine how many rDNS requests are associated with each owner).

[0307] The diagram at 1316 shows... Figure 13A The processing described in the text and Figure 13B The connections between the processes described in the document.

[0308] At 1318, for each unique IP address identified at 1310, processing steps 1320, 1322, and 1322 can be performed.

[0309] At 1320, enhanced data associated with a unique IP address can be used to determine the number of rDNS requests involving that unique IP address.

[0310] At 1322, a comparison can be performed between the number of rDNS requests involving unique IP addresses (determined at 1320) and the IP-level threshold determined at 1302. Accordingly, it can be determined whether the number of rDNS requests involving unique IP addresses is higher or lower than the IP-level threshold (e.g., a certain amount).

[0311] At point 1324, a determination is made regarding whether the IP-level threshold has been exceeded for a unique IP address.

[0312] If the IP-level threshold is not exceeded, processing can continue, such as checking if another threshold has been exceeded (e.g., IP-level threshold for different IP addresses, fully qualified domain name-level threshold, owner-level threshold), and / or continuing to monitor the environment (e.g., performing 1308).

[0313] If the IP level threshold is exceeded, then 1342 can be executed.

[0314] At 1342, it is determined whether a threshold should be exempted (e.g., IP-level threshold, FQDN-level threshold, owner-level threshold). In some embodiments, exceeding the threshold can be exempted if the corresponding IP address, FQDN, and / or owner is in the allowed list and / or not in the blocked list.

[0315] In some embodiments, if a VCN is part of a "listening station" or "signal gathering" honeypot, exceeding a threshold can be exempted, and that VCN can then be considered a control group compared to other VCNs that may have been pre-programmed with alarm and response actions.

[0316] In some embodiments, observed network activity can be exempted if it conforms to a pattern of interest (e.g., predictable traversal of prefixes, known attacker profiling of hosts associated with the same namespace).

[0317] In some embodiments, the conditions for exempting observed network activity can be set based on information about the consequences of blocking the observed network activity (e.g., using a honeypot to gather information about observed network activity may be considered more valuable than blocking the activity). Therefore, by exempting certain network activities that exceed a threshold, the exempted network activities can allow for further insights to help manage subsequent network activities.

[0318] If a valid exemption exceeding the threshold is determined at 1342, 1344 can return to the process of checking whether another threshold has been exceeded (e.g., IP level threshold for different IP addresses, FQDN level threshold, owner level threshold), and / or continue monitoring the environment (e.g., execute 1308).

[0319] If it is determined at 1342 that there is no valid exemption exceeding the threshold, one or more actions can be performed at 1348. In some embodiments, the actions may be generating a report, transmitting a report, configuring a firewall, taking a system (e.g., VCN, zone, CSPI, host machine) offline, adding to a block list, and / or configuring packets to be dropped.

[0320] After any action is performed, a check can be performed to see if another threshold has been exceeded (e.g., IP level threshold for different IP addresses, FQDN level threshold, owner level threshold), and / or the process can continue by monitoring the environment (e.g., performing 1308).

[0321] At 1326, for each unique FQDN identified at 1312, processes 1328, 1330, and 1332 can be performed.

[0322] At point 1328, the number of rDNS requests involving a unique FQDN can be determined using the enhanced data associated with the FQDN. The FQDN can be included in the rDNS request response.

[0323] At 1330, a comparison can be performed between the number of rDNS requests involving FQDNs (determined at 1328) and the FQDN level threshold determined at 1304. Accordingly, it can be determined whether the number of rDNS requests involving unique FQDNs is higher or lower than the FQDN level threshold (e.g., a certain amount).

[0324] At point 1332, a determination is made as to whether the unique FQDN has exceeded the FQDN level threshold.

[0325] If the FQDN level threshold is not exceeded, processing can continue, such as checking if another threshold has been exceeded (e.g., FQDN level threshold for different FQDNs, IP address level threshold, owner level threshold), and / or continuing to monitor the environment (e.g., performing 1308).

[0326] If the FQDN level threshold is exceeded, step 1342 can be executed. The processing performed at step 1342 has already been described above.

[0327] At 1334, for each unique owner identified at 1314, processes 1336, 1338, and 1340 can be performed.

[0328] At position 1336, the number of rDNS requests involving a unique owner can be determined using enhanced data associated with that owner. The owner can be associated with the FQDN included in the rDNS request response and / or the IP address that serves as the subject of the rDNS request.

[0329] At 1338, a comparison can be performed between the number of rDNS requests involving the owner (determined at 1336) and the owner-level threshold determined at 1314. Accordingly, it can be determined whether the number of rDNS requests involving the sole owner is higher or lower than the owner-level threshold (e.g., a certain amount).

[0330] At point 1340, a determination is made regarding whether the sole owner has exceeded the owner-level threshold.

[0331] If the owner-level threshold is not exceeded, processing can continue, such as checking if another threshold is exceeded (e.g., owner-level threshold for different owners, IP address-level threshold, FQDN-level threshold), and / or continuing to monitor the environment (e.g., performing 1308).

[0332] If the owner-level threshold is exceeded, step 1342 can be executed. The processing performed at step 1342 has already been described above.

[0333] The systems described herein may include multiple systems communicatively coupled to each other via one or more communication networks. Furthermore, the techniques illustrated are merely examples and are not intended to unduly limit the scope of the claimed embodiments. Many variations, substitutions, and modifications are possible with respect to the illustrated and described techniques. The depicted systems, subsystems, and other components may be implemented using software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective systems, using hardware, or a combination thereof. The software may be stored on a non-transitory storage medium (e.g., a memory device).

[0334] As indicated above, Infrastructure as a Service (IaaS) is a specific type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, cloud providers can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, IaaS providers can also provision a wide variety of services to accompany those infrastructure components (example services include billing software, monitoring software, logging software, load balancing software, clustering software, etc.). Therefore, because these services can be policy-driven, IaaS users can implement policies to drive load balancing to maintain application availability and performance.

[0335] In some cases, IaaS customers can access resources and services over a wide area network (WAN) (such as the Internet) and can use the cloud provider's services to install the remaining elements of the application stack. For example, a user can log in to the IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create buckets for workloads and backups, and even install enterprise software into the VM. The customer can then use the provider's services to perform various functions, including balancing network traffic, troubleshooting application problems, monitoring performance, and managing disaster recovery.

[0336] In most cases, cloud computing models will require the involvement of cloud providers. Cloud providers can, but do not need to, be third-party providers specializing in (e.g., offering, renting, or selling) IaaS services. Entities can also choose to deploy private clouds, becoming providers of their own infrastructure services.

[0337] In some examples, IaaS deployment is the process of placing a new application or a new version of an application onto a prepared application server or similar. It may also include the process of preparing the server (e.g., installing libraries, daemons, etc.). This is typically managed by the cloud provider under a hypervisor layer (e.g., servers, storage devices, network hardware, and virtualization). Therefore, the customer can be responsible for disposal (OS), middleware, and / or application deployment (e.g., on self-service virtual machines that can be spun up on demand).

[0338] In some examples, IaaS provisioning can refer to acquiring a computer or virtual host for use, and even installing the necessary libraries or services on it. In most cases, deployment does not include provisioning, and provisioning may need to be performed first.

[0339] In some cases, there are two distinct challenges to IaaS provisioning. First, there is the initial challenge of provisioning the initial set of infrastructure before anything can run. Second, there is the challenge of evolving the existing infrastructure after everything has been provisioned (e.g., adding new services, changing services, removing services, etc.). In some cases, these challenges can be addressed by enabling the configuration of the infrastructure to be declaratively defined. In other words, the infrastructure (e.g., what components are needed and how they interact) can be defined by one or more configuration files. Therefore, the overall topology of the infrastructure can be declaratively described (e.g., which resources depend on which resources and how each of them works together). In some cases, after the topology is defined, workflows for creating and / or managing the different components described in the configuration files can be generated.

[0340] In some examples, the infrastructure can have many interconnected elements. For example, there may be one or more Virtual Private Clouds (VPCs) (e.g., potential on-demand pools of configurable and / or shared computing resources), also known as the core network. In some examples, there may also be one or more inbound / outbound traffic group rules, provisioned to define how inbound and / or outbound traffic will be structured for the network, and one or more virtual machines (VMs). Other infrastructure elements, such as load balancers, databases, etc., may also be provisioned. As more infrastructure elements are expected and / or added, the infrastructure can evolve incrementally.

[0341] In some cases, continuous deployment techniques can be used to enable the deployment of infrastructure code across various virtual computing environments. Additionally, the described techniques can enable infrastructure management within these environments. In some examples, service teams may write code that they expect to deploy to one or more (but often more) different production environments (e.g., across different geographical locations, sometimes across the world). However, in some examples, the infrastructure on which the code will be deployed must first be established. In some instances, provisioning can be done manually, provisioning tools can be used to provision resources, and / or, once the infrastructure is provisioned, deployment tools can be used to deploy the code.

[0342] Figure 14 This is a block diagram 1600 illustrating an example pattern of an IaaS architecture according to at least one embodiment. Service operator 1602 may be communicatively coupled to a secure host lease 1604, which may include a virtual cloud network (VCN) 1606 and a secure host subnet 1608. In some examples, service operator 1602 may use one or more client computing devices, which may be portable handheld devices (e.g., iPhone®, cellular phone, iPad®, computing tablet, personal digital assistant (PDA)) or wearable devices (e.g., Google Glass® head-mounted display), running software (such as Microsoft Windows Mobile®) and / or a wide variety of mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 8, Palm OS, etc.), and enabled for the Internet, email, SMS, Blackberry®, or other communication protocols. Alternatively, client computing devices may be general-purpose personal computers, for example including personal computers and / or laptops running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems. The client computing device can be a workstation computer running any of the various commercial UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, such as, for example, Google Chrome OS). Alternatively or additionally, the client computing device can be any other electronic device capable of communicating over a network that can access the VCN 1606 and / or the Internet, such as a thin client computer, an Internet-enabled gaming system (e.g., a Microsoft Xbox game console with or without Kinect® gesture input), and / or a personal messaging device.

[0343] VCN 1606 may include a local peering gateway (LPG) 1610, which may be communicatively coupled to a secure shell (SSH) VCN 1612 via LPG 1610 included in SSH VCN 1612. SSH VCN 1612 may include an SSH subnet 1614, and SSH VCN 1612 may be communicatively coupled to a control plane VCN 1616 via LPG 1610 included in control plane VCN 1616. Furthermore, SSH VCN 1612 may be communicatively coupled to a data plane VCN 1618 via LPG 1610. Control plane VCN 1616 and data plane VCN 1618 may be included in a service lease 1619 that may be owned and / or operated by an IaaS provider.

[0344] The control plane VCN 1616 may include a control plane demilitarized zone (DMZ) layer 1620, which acts as a portion of the perimeter network (e.g., a corporate network between an intranet and an external network). DMZ-based servers can have limited liability and help keep breaches contained. Additionally, the DMZ layer 1620 may include one or more load balancer (LB) subnets 1622, a control plane application layer 1624 that may include one or more application subnets 1626, and a control plane data layer 1628 that may include one or more database (DB) subnets 1630 (e.g., one or more front-end DB subnets and / or one or more back-end DB subnets). One or more LB subnets 1622 contained in the control plane DMZ layer 1620 may be communicatively coupled to one or more application subnets 1626 contained in the control plane application layer 1624 and an Internet gateway 1634 that may be contained in the control plane VCN 1616. The application subnets 1626 may be communicatively coupled to one or more DB subnets 1630 and service gateways 1636 and Network Address Translation (NAT) gateways 1638 contained in the control plane data layer 1628. The control plane VCN 1616 may include service gateway 1636 and NAT gateway 1638.

[0345] Control plane VCN 1616 may include a data plane mirroring application layer 1640, which may include one or more application subnets 1626. The one or more application subnets 1626 included in the data plane mirroring application layer 1640 may include a virtual network interface controller (VNIC) 1642, which can execute a compute instance 1644. The compute instance 1644 may communicatively couple the one or more application subnets 1626 of the data plane mirroring application layer 1640 to the one or more application subnets 1626 that may be included in the data plane application layer 1646.

[0346] Data plane VCN 1618 may include a data plane application layer 1646, a data plane DMZ layer 1648, and a data plane data layer 1650. Data plane DMZ layer 1648 may include one or more application subnets 1626 communicatively coupled to data plane application layer 1646 and one or more LB subnets 1622 of Internet gateway 1634 of data plane VCN 1618. One or more application subnets 1626 may be communicatively coupled to service gateway 1636 and NAT gateway 1638 of data plane VCN 1618. Data plane data layer 1650 may also include one or more DB subnets 1630 communicatively coupled to one or more application subnets 1626 of data plane application layer 1646.

[0347] Internet gateway 1634 of control plane VCN 1616 and data plane VCN 1618 can communicatively couple to metadata management service 1652, which can communicatively couple to public internet 1654. Public internet 1654 can communicatively couple to NAT gateway 1638 of control plane VCN 1616 and data plane VCN 1618. Service gateway 1636 of control plane VCN 1616 and data plane VCN 1618 can communicatively couple to cloud service 1656.

[0348] In some examples, the service gateway 1636 of the control plane VCN 1616 or the data plane VCN 1618 can make application programming interface (API) calls to the cloud service 1656 without traversing the public internet 1654. API calls from the service gateway 1636 to the cloud service 1656 can be unidirectional: the service gateway 1636 can make API calls to the cloud service 1656, and the cloud service 1656 can send the requested data to the service gateway 1636. However, the cloud service 1656 may not initiate API calls to the service gateway 1636.

[0349] In some examples, secure host lease 1604 can be directly connected to service lease 1619, which would otherwise be isolated. Secure host subnet 1608 can communicate with SSH subnet 1614 via LPG 1610, which enables bidirectional communication over otherwise isolated systems. Connecting secure host subnet 1608 to SSH subnet 1614 grants secure host subnet 1608 access to other entities within service lease 1619.

[0350] Control plane VCN 1616 may allow users of service lease 1619 to establish or otherwise provision desired resources. The desired resources provisioned in control plane VCN 1616 may be deployed or otherwise used in data plane VCN 1618. In some examples, control plane VCN 1616 may be isolated from data plane VCN 1618, and the data plane mirror application layer 1640 of control plane VCN 1616 may communicate with the data plane application layer 1646 of data plane VCN 1618 via a VNIC 1642 that may be included in both the data plane mirror application layer 1640 and the data plane application layer 1646.

[0351] In some examples, a user or client of the system may make a request (e.g., a create, read, update, or delete (CRUD) operation) via the public internet 1654, which can then forward the request to the metadata management service 1652. The metadata management service 1652 can then forward the request to the control plane VCN 1616 via internet gateway 1634. The request may be received by one or more LB subnets 1622 contained in the control plane DMZ layer 1620. The LB subnets 1622 can determine that the request is valid, and in response to this determination, they can forward the request to one or more application subnets 1626 contained in the control plane application layer 1624. If the request is authenticated and requires a call to the public internet 1654, the call to the public internet 1654 can be forwarded to a NAT gateway 1638 that can make the call to the public internet 1654. The request may expect that the metadata to be stored can be stored in one or more DB subnets 1630.

[0352] In some examples, the data plane mirroring application layer 1640 can facilitate direct communication between the control plane VCN 1616 and the data plane VCN 1618. For example, it may be desirable to apply configuration changes, updates, or other suitable modifications to resources contained in the data plane VCN 1618. Through VNIC 1642, the control plane VCN 1616 can communicate directly with the resources contained in the data plane VCN 1618, and thereby perform configuration changes, updates, or other suitable modifications to the resources contained in the data plane VCN 1618.

[0353] In some embodiments, the control plane VCN 1616 and data plane VCN 1618 may be included in service lease 1619. In this case, the system's users or customers may not own or operate the control plane VCN 1616 or data plane VCN 1618. Conversely, the IaaS provider may own or operate both the control plane VCN 1616 and data plane VCN 1618, both of which may be included in service lease 1619. This embodiment can enable network isolation that can prevent users or customers from interacting with resources of other users or other customers. Furthermore, this embodiment can allow the system's users or customers to privately store databases without relying on the public Internet 1654, which may not have the desired level of threat prevention.

[0354] In other embodiments, one or more LB subnets 1622 included in the control plane VCN 1616 may be configured to receive signals from the service gateway 1636. In this embodiment, the control plane VCN 1616 and the data plane VCN 1618 may be configured to be invoked by the IaaS provider's customers without invoking the public internet 1654. The IaaS provider's customers may expect this embodiment because the database(s) used by the customer can be controlled by the IaaS provider and can be stored on a service lease 1619 that can be isolated from the public internet 1654.

[0355] Figure 15 This is a block diagram 1700 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 1702 (e.g., Figure 16 The service provider 1602) can communicatively couple to the secure host lease 1704 (e.g., Figure 16 Secure hosting lease 1604), the secure hosting lease may include a Virtual Cloud Network (VCN) 1706 (e.g., Figure 16 VCN1606) and Secure Host Subnet 1708 (e.g., Figure 16 The secure host subnet 1608). VCN 1706 may include a local peering gateway (LPG) 1710 (e.g., Figure 16 The LPG 1610 can be communicatively coupled to the Secure Shell (SSH) VCN 1712 (e.g., LPG 1610) contained in the SSH VCN 1712. Figure 16 (SSH) VCN 1612). SSH VCN 1712 can include SSH subnet 1714 (e.g., Figure 16 SSH subnet 1614), and SSH VCN 1712 can be communicatively coupled to control plane VCN 1716 via LPG 1710 contained in control plane VCN 1716 (e.g., Figure 16 Control plane VCN 1716. Control plane VCN 1716 may be included in service lease 1719 (e.g., Figure 16 In the service lease 1619), and the data plane VCN 1718 (e.g., Figure 16 The data plane VCN 1618 can be included in a customer lease 1721 that can be owned or operated by the system's user or customer.

[0356] The control plane VCN 1716 may include one or more LB subnets 1722 (e.g., Figure 16 The control plane DMZ layer 1720 of (one or more) LB subnets 1622) (e.g., Figure 16 The control plane DMZ layer 1620 may include one or more application subnets 1726 (e.g., Figure 16 The control plane application layer 1724 of (one or more) application subnets 1626 (e.g., Figure 16 The control plane application layer 1624 may include one or more database (DB) subnets 1730 (e.g., similar to...). Figure 16 The control plane data layer 1728 of (one or more) DB subnets 1630 (e.g., Figure 16 The control plane data layer 1628). One or more LB subnets 1722 contained in the control plane DMZ layer 1720 can be communicatively coupled to one or more application subnets 1726 contained in the control plane application layer 1724 and an Internet gateway 1734 that can be contained in the control plane VCN 1716 (e.g., Figure 16 Internet gateway 1634), and application subnet(s) 1726 can communicatively couple to DB subnet(s) 1730 and service gateway 1736 contained in control plane data layer 1728 (e.g., Internet gateway 1634), and application subnet(s) 1726 can communicatively couple to DB subnet(s) 1730 and service gateway(s) ... Figure 16 Service gateway 1636) and Network Address Translation (NAT) gateway 1738 (e.g., Figure 16 (NAT gateway 1638). The control plane VCN 1716 may include the service gateway 1736 and the NAT gateway 1738.

[0357] The control plane VCN 1716 may include a data plane mirror of the application layer 1740 (e.g., Figure 16 The data plane mirroring application layer 1640 may include one or more application subnets 1726. The application subnets 1726 included in the data plane mirroring application layer 1740 may include instances 1744 capable of performing computations (e.g., similar to...). Figure 16 The virtual network interface controller (VNIC) 1742 (e.g., the VNIC of 1642) of the computing instance 1644. The computing instance 1744 may facilitate the mirroring of one or more application subnets 1726 of the data plane application layer 1740 and may be included in the data plane application layer 1746 (e.g., Figure 16 Communication between one or more application subnets 1726 in the data plane application layer 1646 via VNIC 1742 contained in the data plane mirror application layer 1740 and VNIC 1742 contained in the data plane application layer 1746.

[0358] The Internet gateway 1734, included in the control plane VCN 1716, can be communicatively coupled to the metadata management service 1752 (e.g., Figure 16 Metadata management service 1652), which can communicatively couple to the public Internet 1754 (e.g., Figure 16 The public internet 1754 can communicatively couple to a NAT gateway 1738 contained in a control plane VCN 1716. The service gateway 1736 contained in the control plane VCN 1716 can communicatively couple to a cloud service 1756 (e.g., ...). Figure 16 Cloud services (1656).

[0359] In some examples, data plane VCN 1718 may be included in customer lease 1721. In this case, the IaaS provider may provide control plane VCN 1716 for each customer, and the IaaS provider may establish a unique compute instance 1744 for each customer, included in service lease 1719. Each compute instance 1744 may allow communication between control plane VCN 1716 included in service lease 1719 and data plane VCN 1718 included in customer lease 1721. Compute instance 1744 may allow resources provisioned in control plane VCN 1716 included in service lease 1719 to be deployed or otherwise used in data plane VCN 1718 included in customer lease 1721.

[0360] In other examples, an IaaS provider's customer may have a database residing in customer lease 1721. In this example, control plane VCN 1716 may include a data plane mirroring application layer 1740, which may include one or more application subnets 1726. Data plane mirroring application layer 1740 may reside in data plane VCN 1718, but it may not reside in data plane VCN 1718. That is, data plane mirroring application layer 1740 may have access to customer lease 1721, but it may not reside in data plane VCN 1718 or be owned or operated by an IaaS provider's customer. Data plane mirroring application layer 1740 may be configured to make calls to data plane VCN 1718, but it may not be configured to make calls to any entity contained in control plane VCN 1716. Customers may expect to deploy or otherwise use resources provided in the control plane VCN 1716 in the data plane VCN 1718, and the data plane mirroring application layer 1740 can facilitate the customer's desired deployment or other use of resources.

[0361] In some embodiments, an IaaS provider's customer may apply filters to data plane VCN 1718. In this embodiment, the customer may determine what data plane VCN 1718 can access, and the customer may restrict access from data plane VCN 1718 to the public Internet 1754. The IaaS provider may not be able to apply filters or otherwise control data plane VCN 1718's access to any external networks or databases. Applying filters and controls by the customer to data plane VCN 1718, which is included in customer lease 1721, can help isolate data plane VCN 1718 from other customers and from the public Internet 1754.

[0362] In some embodiments, cloud service 1756 can be invoked by service gateway 1736 to access services that may not exist on public internet 1754, control plane VCN 1716, or data plane VCN 1718. The connection between cloud service 1756 and control plane VCN 1716 or data plane VCN 1718 may not be active or continuous. Cloud service 1756 may reside on different networks owned or operated by an IaaS provider. Cloud service 1756 may be configured to receive calls from service gateway 1736 and may be configured not to receive calls from public internet 1754. Some cloud services 1756 may be isolated from other cloud services 1756, and control plane VCN 1716 may be isolated from cloud services 1756, meaning the cloud service may not be in the same region as control plane VCN 1716. For example, control plane VCN 1716 may be located in "Region 1," and cloud service "Deployment 16" may be located in both Region 1 and "Region 2." If a call to deployment 16 is made by a service gateway 1736 contained in a control plane VCN 1716 located in region 1, the call can be forwarded to deployment 16 in region 1. In this example, the control plane VCN 1716 or deployment 16 in region 1 may be non-communicatively coupled to deployment 16 in region 2, or may otherwise communicate with deployment 16 in region 2.

[0363] Figure 16 This is a block diagram 1800 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 1802 (e.g., Figure 16 The service provider 1602) can communicatively couple to the secure host lease 1804 (e.g., Figure 16 Secure hosting lease 1604), the secure hosting lease may include a Virtual Cloud Network (VCN) 1806 (e.g., Figure 16 VCN1606) and Secure Host Subnet 1808 (e.g., Figure 16 The secure host subnet 1608). VCN 1806 can include LPG 1810 (e.g., Figure 16 The LPG 1610 can be communicatively coupled to the SSH VCN 1812 via the LPG 1810 included in the SSH VCN 1812 (e.g., Figure 16 SSH VCN 1612). SSH VCN 1812 can include SSH subnet 1814 (e.g., Figure 16 SSH subnet 1614), and SSH VCN 1812 can be communicatively coupled to control plane VCN 1816 via LPG 1810 contained in control plane VCN 1816 (e.g., Figure 16 The control plane VCN 1616) and the LPG 1810 communicatively coupled to the data plane VCN 1818 (e.g., via the control plane VCN 1616) and the data plane VCN 1818 via the LPG 1810 contained in the data plane VCN 1818. Figure 16 Data plane 1618). Control plane VCN 1816 and data plane VCN 1818 may be included in service lease 1819 (e.g., Figure 16 In the service rental (1619).

[0364] The control plane VCN 1816 may include one or more load balancer (LB) subnets 1822 (e.g., Figure 16 The control plane DMZ layer 1820 of (one or more) LB subnets 1622 (e.g., Figure 16 The control plane DMZ layer 1620 may include one or more application subnets 1826 (e.g., similar to...). Figure 16 The control plane application layer 1824 of (one or more) application subnets 1626 (e.g., Figure 16 The control plane application layer 1624 may include (one or more) control plane data layers 1828 of the DB subnet 1830 (e.g., Figure 16 The control plane data layer 1628). One or more LB subnets 1822 contained in the control plane DMZ layer 1820 can be communicatively coupled to one or more application subnets 1826 contained in the control plane application layer 1824 and an Internet gateway 1834 that can be contained in the control plane VCN 1816 (e.g., Figure 16 Internet gateway 1634), and application subnet(s) 1826 can communicatively couple to DB subnet(s) 1830 contained in control plane data layer 1828 and service gateway 1836 (e.g., Figure 16 The service gateway) and Network Address Translation (NAT) gateway 1838 (e.g., Figure 16 (NAT gateway 1638). The control plane VCN 1816 may include the service gateway 1836 and the NAT gateway 1838.

[0365] The data plane VCN 1818 may include the data plane application layer 1846 (e.g., Figure 16 Data plane application layer 1646), data plane DMZ layer 1848 (e.g., Figure 16 Data plane DMZ layer 1648), and data plane data layer 1850 (e.g., Figure 16 The data plane data layer 1650. The data plane DMZ layer 1848 may include one or more LB subnets 1822, which may be communicatively coupled to one or more trusted application subnets 1860 and one or more untrusted application subnets 1862 of the data plane application layer 1846, and an Internet gateway 1834 contained in the data plane VCN 1818. One or more trusted application subnets 1860 may be communicatively coupled to a service gateway 1836 contained in the data plane VCN 1818, a NAT gateway 1838 contained in the data plane VCN 1818, and one or more DB subnets 1830 contained in the data plane data layer 1850. One or more untrusted application subnets 1862 may be communicatively coupled to a service gateway 1836 contained in the data plane VCN 1818 and one or more DB subnets 1830 contained in the data plane data layer 1850. The data plane data layer 1850 may include one or more DB subnets 1830, which may be communicatively coupled to a service gateway 1836 contained in the data plane VCN 1818.

[0366] One or more untrusted application subnets 1862 may include one or more primary VNICs 1864(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 1866(1)-(N). Each tenant VM 1866(1)-(N) may be communicatively coupled to a corresponding application subnet 1867(1)-(N) that may be contained in a corresponding container egress VCN 1868(1)-(N), which may be contained in a corresponding customer lease 1870(1)-(N). The corresponding secondary VNIC 1872(1)-(N) may facilitate communication between one or more untrusted application subnets 1862 contained in a data plane VCN 1818 and application subnets contained in a container egress VCN 1868(1)-(N). Each container egress VCN 1868(1)-(N) may include a NAT gateway 1838 that can be communicatively coupled to the public Internet 1854 (e.g., Figure 16 The public internet (1654).

[0367] Internet gateway 1834, contained in control plane VCN 1816 and data plane VCN 1818, can be communicatively coupled to metadata management service 1852 (e.g., Figure 16 A metadata management system 1652 is provided, which can communicatively couple to the public internet 1854. The public internet 1854 can communicatively couple to a NAT gateway 1838 contained in a control plane VCN 1816 and a data plane VCN 1818. A service gateway 1836 contained in a control plane VCN 1816 and a data plane VCN 1818 can communicatively couple to a cloud service 1856.

[0368] In some embodiments, the data plane VCN 1818 can be integrated with the customer-leased 1870. This integration may be useful or desired by the IaaS provider's customers in certain situations, such as when support might be expected when executing code. Customers may provide code to run, which may be destructive, may communicate with other customer resources, or may otherwise cause adverse effects. In response, the IaaS provider may determine whether to run the code provided by the customer.

[0369] In some examples, an IaaS provider's customer can grant temporary network access to the IaaS provider and request functionality to be attached to the data plane application layer 1846. The code running this functionality can execute in VMs 1866(1)-(N), and the code may not be configured to run anywhere else on the data plane VCN 1818. Each VM 1866(1)-(N) can be connected to a customer lease 1870. The corresponding container 1871(1)-(N) contained in VMs 1866(1)-(N) can be configured to run the code. In this case, dual isolation can exist (e.g., container 1871(1)-(N) runs the code, where container 1871(1)-(N) can be contained at least in VM 1866(1)-(N), which is contained in one or more untrusted application subnets 1862), which can help prevent incorrect or otherwise unintended code from corrupting the IaaS provider's network or the networks of different customers. Containers 1871(1)-(N) may be communicatively coupled to customer lease 1870 and may be configured to send or receive data from customer lease 1870. Containers 1871(1)-(N) may not be configured to send or receive data from any other entity in the data plane VCN 1818. After the running code has completed, the IaaS provider may terminate or otherwise dispose of containers 1871(1)-(N).

[0370] In some embodiments, one or more trusted application subnets 1860 may run code owned or operated by the IaaS provider. In this embodiment, one or more trusted application subnets 1860 may be communicatively coupled to one or more DB subnets 1830 and configured to perform CRUD operations in one or more DB subnets 1830. One or more untrusted application subnets 1862 may be communicatively coupled to one or more DB subnets 1830, but in this embodiment, one or more untrusted application subnets may be configured to perform read operations in one or more DB subnets 1830. Containers 1871(1)-(N) that may be contained in each customer's VM 1866(1)-(N) and may run code from the customer may not be communicatively coupled to one or more DB subnets 1830.

[0371] In other embodiments, the control plane VCN 1816 and the data plane VCN 1818 may be coupled without direct communication. In this embodiment, direct communication between the control plane VCN 1816 and the data plane VCN 1818 may not exist. However, communication can occur indirectly through at least one method. An LPG 1810, established by an IaaS provider, can facilitate communication between the control plane VCN 1816 and the data plane VCN 1818. In another example, either the control plane VCN 1816 or the data plane VCN 1818 can make a call to the cloud service 1856 via the service gateway 1836. For example, a call from the control plane VCN 1816 to the cloud service 1856 may include a request for a service that can communicate with the data plane VCN 1818.

[0372] Figure 17 This is a block diagram 1900 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 1902 (e.g., Figure 16 The service provider 1602) can communicatively couple to the secure host lease 1904 (e.g., Figure 16 Secure hosting lease 1604), the secure hosting lease may include a Virtual Cloud Network (VCN) 1906 (e.g., Figure 16 VCN1606) and Secure Host Subnet 1908 (e.g., Figure 16 The secure host subnet 1608). VCN 1906 can include LPG 1910 (e.g., Figure 16 The LPG 1610 can be communicatively coupled to the SSH VCN 1912 via the LPG 1910 included in the SSH VCN 1912 (e.g., Figure 16 SSH VCN 1612). SSH VCN 1912 can include SSH subnet 1914 (e.g., Figure 16 SSH subnet 1614), and SSH VCN 1912 can be communicatively coupled to control plane VCN 1916 via LPG 1910 contained in control plane VCN 1916 (e.g., Figure 16 The control plane VCN 1616) and the LPG 1910 communicatively coupled to the data plane VCN 1918 (e.g., via the control plane VCN 1616) and the data plane VCN 1918 via the LPG 1910 contained in the data plane VCN 1918. Figure 16 Data plane 1916. Control plane VCN 1916 and data plane VCN 1918 may be contained in service lease 1919 (e.g., Figure 16 In the service rental (1619).

[0373] The control plane VCN 1916 may include (one or more) LB subnets 1922 (e.g., Figure 16 The control plane DMZ layer 1920 of (one or more) LB subnets 1622) (e.g., Figure 16 The control plane DMZ layer 1620), may include one or more application subnets 1926 (e.g., Figure 16 The control plane application layer 1924 of (one or more) application subnets 1626 (e.g., Figure 16 The control plane application layer 1624 may include one or more DB subnets 1930 (e.g., Figure 16 The control plane data layer 1928 of (one or more) DB subnets 1830 (e.g., Figure 16 The control plane data layer 1628). One or more LB subnets 1922 contained in the control plane DMZ layer 1920 can be communicatively coupled to one or more application subnets 1926 contained in the control plane application layer 1924 and an Internet gateway 1934 that can be contained in the control plane VCN 1916 (e.g., Figure 16 Internet gateway 1634), and application subnet(s) 1926 can communicatively couple to DB subnet(s) 1930 contained in control plane data layer 1928 and to service gateway 1936 (e.g., Figure 16 The service gateway) and Network Address Translation (NAT) gateway 1938 (e.g., Figure 16 (NAT gateway 1638). The control plane VCN 1916 may include the service gateway 1936 and the NAT gateway 1938.

[0374] Data plane VCN 1918 may include data plane application layer 1946 (e.g., Figure 16 Data plane application layer 1646), data plane DMZ layer 1948 (e.g., Figure 16 Data plane DMZ layer 1648), and data plane data layer 1950 (e.g., Figure 16 The data plane data layer 1650). The data plane DMZ layer 1948 may include one or more LB subnets 1922, which may be communicatively coupled to one or more trusted application subnets 1960 of the data plane application layer 1946 (e.g., Figure 16 (one or more) trusted application subnets (1860) and (one or more) untrusted application subnets (1962) (e.g., Figure 16 The data plane includes one or more untrusted application subnets 1962 and an Internet gateway 1934 contained in data plane VCN 1918. One or more trusted application subnets 1960 may communicatively couple to a service gateway 1936 contained in data plane VCN 1918, a NAT gateway 1938 contained in data plane VCN 1918, and one or more DB subnets 1930 contained in data plane data layer 1950. One or more untrusted application subnets 1962 may communicatively couple to a service gateway 1936 contained in data plane VCN 1918 and one or more DB subnets 1930 contained in data plane data layer 1950. Data plane data layer 1950 may include one or more DB subnets 1930 that may communicatively couple to a service gateway 1936 contained in data plane VCN 1918.

[0375] One or more untrusted application subnets 1962 may include a primary VNIC 1964(1)-(N) that may be communicatively coupled to tenant virtual machines (VMs) 1966(1)-(N) residing within one or more untrusted application subnets 1962. Each tenant VM 1966(1)-(N) may run code in a corresponding container 1967(1)-(N) and may be communicatively coupled to an application subnet 1926 that may be contained in a data plane application layer 1946, which may be contained in a container egress VCN 1968. A corresponding secondary VNIC 1972(1)-(N) may facilitate communication between one or more untrusted application subnets 1962 contained in a data plane VCN 1918 and the application subnets contained in a container egress VCN 1968. The container egress VCN may include a NAT gateway 1938 that may be communicatively coupled to the public internet 1954 (e.g., Figure 16 The public internet (1654).

[0376] Internet gateway 1934, contained in control plane VCN 1916 and data plane VCN 1918, can communicatively couple to metadata management service 1952 (e.g., Figure 16 A metadata management system 1652), which provides metadata management services, can communicatively couple to the public internet 1954. The public internet 1954 can communicatively couple to a NAT gateway 1938 contained in a control plane VCN 1916 and a data plane VCN 1918. A service gateway 1936 contained in a control plane VCN 1916 and a data plane VCN 1918 can communicatively couple to a cloud service 1956.

[0377] In some examples, by Figure 17 The block diagram of the 1900 architecture can be viewed as a pattern derived from... Figure 16 The architecture illustrated in block diagram 1800 is an exception to the pattern, and may be what the IaaS provider's customers expect if the IaaS provider cannot communicate directly with the customer (e.g., in a disconnected region). The corresponding container 1967(1)-(N) contained in each customer's VM 1966(1)-(N) can be accessed by the customer in real time. Container 1967(1)-(N) can be configured to make calls to the corresponding secondary VNIC 1972(1)-(N) contained in one or more application subnets 1926 of the data plane application layer 1946, which may be contained in the container egress VCN 1968. The secondary VNIC 1972(1)-(N) can forward the calls to a NAT gateway 1938, which can forward the calls to the public internet 1954. In this example, containers 1967(1)-(N), which can be accessed by clients in real time, can be isolated from the control plane VCN 1916 and from other entities contained in the data plane VCN 1918. Containers 1967(1)-(N) can also be isolated from resources from other clients.

[0378] In other examples, a client may use container 1967(1)-(N) to invoke cloud service 1956. In this example, the client may run code requesting a service from cloud service 1956 within container 1967(1)-(N). Container 1967(1)-(N) may forward the request to a secondary VNIC 1972(1)-(N), which may forward the request to a NAT gateway, which may forward the request to the public internet 1954. The public internet 1954 may forward the request via internet gateway 1934 to one or more LB subnets 1922 contained in control plane VCN 1916. In response to determining that the request is valid, one or more LB subnets may forward the request to one or more application subnets 1926, which may forward the request to cloud service 1956 via service gateway 1936.

[0379] It should be understood that the IaaS architectures 1600, 1700, 1800, and 1900 depicted in the accompanying drawings may have other components besides those depicted. Furthermore, the embodiments shown in the drawings are merely some examples of cloud infrastructure systems that can be incorporated into embodiments of this disclosure. In some other embodiments, the IaaS system may have more or fewer components than shown in the drawings, may combine two or more components, or may have different configurations or arrangements of components.

[0380] In some embodiments, the IaaS system described herein may include a suite of application, middleware, and database service offerings delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such an IaaS system is the Oracle Cloud Infrastructure (OCI) provided by this assignee.

[0381] Figure 18 An example computer system 2000, in which various embodiments can be implemented, is illustrated. System 2000 can be used to implement any computer system described above. As shown in the figure, computer system 2000 includes a processing unit 2004 that communicates with a plurality of peripheral subsystems via a bus subsystem 2002. These peripheral subsystems may include a processing acceleration unit 2006, an I / O subsystem 2008, a storage subsystem 2018, and a communication subsystem 2024. Storage subsystem 2018 includes a tangible computer-readable storage medium 2022 and system memory 2010.

[0382] The bus subsystem 2002 provides a mechanism for enabling the various components and subsystems of the computer system 2000 to communicate with each other as intended. Although the bus subsystem 2002 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. The bus subsystem 2002 can be any of several types of bus architectures, including memory buses or memory controllers, peripheral buses, and local buses using any of a variety of bus architectures. For example, such architectures may include Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses that may be implemented as mezzanine buses manufactured according to the IEEE P1386.1 standard.

[0383] A processing unit 2004, which may be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of a computer system 2000. One or more processors may be included in the processing unit 2004. These processors may include single-core or multi-core processors. In some embodiments, the processing unit 2004 may be implemented as one or more independent processing units 2032 and / or 2034, each including a single-core or multi-core processor. In other embodiments, the processing unit 2004 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0384] In various embodiments, processing unit 2004 can execute various programs in response to program code and can maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed may reside in processor(s) 2004 and / or storage subsystem 2018. With appropriate programming, processor(s) 2004 can provide the various functions described above. Additionally, computer system 2000 may include processing acceleration unit 2006, which may include a digital signal processor (DSP), a dedicated processor, and / or the like.

[0385] The I / O subsystem 2008 may include user interface input devices and user interface output devices. User interface input devices may include keyboards, pointing devices such as mice or trackballs, touchpads or touchscreens integrated into the display, scroll wheels, click wheels, dial pads, buttons, switches, keypads, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may include, for example, motion sensing and / or gesture recognition devices, such as the Microsoft Kinect® motion sensor, which enables users to control and interact with input devices (such as the Microsoft Xbox® 360 game controller) using a natural user interface that employs gestures and verbal commands. User interface input devices may also include eye gesture recognition devices, such as the Google Glass® blink detector, which detects eye activity from the user (e.g., "blinking" when taking a photo and / or making menu selections) and translates the eye gesture into input on the input device (e.g., Google Glass®). Additionally, user interface input devices may include voice recognition sensing devices that enable users to interact with a voice recognition system (e.g., the Siri® navigator) via voice commands.

[0386] User interface input devices may also include, but are not limited to, 3D mice, joysticks or pointers, game controllers, and graphics tablets, as well as audio / video devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. Additionally, user interface input devices may include, for example, medical imaging input devices such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and medical ultrasound equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards and digital musical instruments.

[0387] User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices. Display subsystems may be cathode ray tubes (CRTs), flat panel devices such as those using liquid crystal displays (LCDs) or plasma displays, projection devices, touchscreens, etc. Generally, the term "output device" is used to encompass all possible types of devices and mechanisms for outputting information from the computer system 2000 to the user or other computers. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0388] Computer system 2000 may include a storage subsystem 2018 that can provide a tangible, non-transitory computer-readable storage medium for storing software and data constructs that provide the functionality of the embodiments described in this disclosure. The software may include programs, code modules, instructions, scripts, etc., which, when executed by one or more cores or processors of processing unit 2004, provide the aforementioned functionality. Storage subsystem 2018 may also provide a repository for storing data used according to this disclosure.

[0389] like Figure 18 As depicted in the example, the storage subsystem 2018 may include various components, including system memory 2010, computer-readable storage medium 2022, and computer-readable storage medium reader 2020. System memory 2010 may store program instructions that can be loaded and executed by processing unit 2004. System memory 2010 may also store data used during the execution of instructions and / or data generated during the execution of program instructions. Various types of programs may be loaded into system memory 2010, including but not limited to client applications, web browsers, middleware applications, relational database management systems (RDBMS), virtual machines, containers, etc.

[0390] System memory 2010 may also store operating system 2016. Examples of operating system 2016 may include various versions of Microsoft Windows®, Apple Macintosh® and / or Linux operating systems, various commercial UNIX® or UNIX-like operating systems (including but not limited to various GNU / Linux operating systems, Google Chrome® OS, etc.) and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® OS, and Palm® OS. In some implementations of computer system 2000 that execute one or more virtual machines, the virtual machine, along with its guest operating system (GOS), may be loaded into system memory 2010 and executed by one or more processors or cores of processing unit 2004.

[0391] Depending on the type of computer system 2000, the system memory 2010 can have different configurations. For example, the system memory 2010 can be volatile memory (such as random access memory (RAM)) and / or non-volatile memory (such as read-only memory (ROM), flash memory, etc.). Different types of RAM configurations can be provided, including static random access memory (SRAM), dynamic random access memory (DRAM), etc. In some implementations, the system memory 2010 may include a basic input / output system (BIOS), which contains basic routines that facilitate the transfer of information between elements within the computer system 2000, such as during startup.

[0392] Computer-readable storage medium 2022 may represent remote, local, fixed and / or removable storage devices and storage media for temporarily and / or more permanently containing and storing computer-readable information used by computer system 2000, including instructions executable by processing unit 2004 of computer system 2000.

[0393] Computer-readable storage media 2022 may include any suitable media known or used in the art, including storage media and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing and / or transmitting information. This may include tangible computer-readable storage media or other tangible computer-readable media such as RAM, ROM, electronically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices.

[0394] For example, computer-readable storage media 2022 may include hard disk drives that read from or write to non-removable non-volatile magnetic media, disk drives that read from or write to removable non-volatile disks, and optical disc drives that read from or write to removable non-volatile optical discs (such as CD ROMs, DVDs, and Blu-ray® discs) or other optical media. Computer-readable storage media 2022 may include, but is not limited to, Zip® drives, flash memory cards, Universal Serial Bus (USB) flash memory drives, Secure Digital (SD) cards, DVD discs, digital video tapes, etc. Computer-readable storage media 2022 may also include solid-state drives (SSDs) based on non-volatile memory (such as flash-based SSDs, enterprise flash drives, solid-state ROMs, etc.), volatile memory-based SSDs (such as solid-state RAM, dynamic RAM, static RAM, DRAM-based SSDs), magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs using a combination of DRAM and flash-based SSDs. Disk drives and their associated computer-readable media can provide non-volatile storage for computer-readable instructions, data structures, program modules and other data for a computer system 2000.

[0395] Machine-readable instructions executable by one or more processors or cores of the processing unit 2004 may be stored on a non-transitory computer-readable storage medium. A non-transitory computer-readable storage medium may include physically tangible memory or storage devices that include volatile memory storage devices and / or non-volatile memory storage devices. Examples of non-transitory computer-readable storage media include magnetic storage media (e.g., magnetic disks or magnetic tapes), optical storage media (e.g., DVDs, CDs), various types of RAM, ROM, or flash memory, hard disk drives, floppy disk drives, removable memory drives (e.g., USB drives), or other types of storage devices.

[0396] The communication subsystem 2024 provides interfaces to other computer systems and networks. The communication subsystem 2024 serves as an interface for receiving data from other systems and sending data to other systems from computer system 2000. For example, the communication subsystem 2024 may enable computer system 2000 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 2024 may include radio frequency (RF) transceiver components, global positioning system (GPS) receiver components, and / or other components for accessing wireless voice and / or data networks (e.g., using cellular phone technology, advanced data network technologies such as 3G, 4G, or EDGE (Enhanced Data Rate Global Evolution), WiFi (IEEE 802.11 series standards, or other mobile communication technologies, or any combination thereof)). In some embodiments, in addition to or instead of the wireless interface, the communication subsystem 2024 may provide wired network connectivity (e.g., Ethernet).

[0397] In some embodiments, the communication subsystem 2024 may also receive input communications on behalf of one or more users who may use the computer system 2000 in the form of structured and / or unstructured data feeds 2026, event streams 2028, event updates 2030, etc.

[0398] For example, the communication subsystem 2024 can be configured to receive data feeds 2026 in real time from users of social networks and / or other communication services, such as Twitter® feeds, Facebook® updates, web feeds such as Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0399] Additionally, the communication subsystem 2024 can also be configured to receive data in the form of a continuous data stream, which may include event streams 2028 and / or event updates 2030 of real-time events. This data may be continuous or unbounded in nature, without a definite end. Examples of applications that generate continuous data may include, for example, sensor data applications, financial stock analysis, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, etc.

[0400] The communication subsystem 2024 can also be configured to output structured and / or unstructured data feeds 2026, event streams 2028, event updates 2030, etc., to one or more databases that can communicate with one or more streaming data source computers coupled to the computer system 2000.

[0401] The computer system 2000 can be one of a variety of types, including handheld portable devices (e.g., iPhone® cellular phones, iPad® computing tablets, PDAs), wearable devices (e.g., Google Glass® head-mounted displays), PCs, workstations, mainframes, kiosks, server racks, or any other data processing system.

[0402] Due to the constantly evolving nature of computers and networks, the description of the computer system 2000 depicted in the figures is intended only as a specific example. Many other configurations with more or fewer components than the system depicted in the figures are possible. For example, custom hardware may also be used and / or specific elements may be implemented in hardware, firmware, software (including applets), or a combination thereof. Furthermore, connectivity with other computing devices, such as network input / output devices, may be employed. Based on the disclosure and teachings provided herein, those skilled in the art will understand other ways and / or methods for implementing the various embodiments.

[0403] Although specific embodiments have been described, various modifications, alterations, alternative constructions, and equivalents are also covered within the scope of this disclosure. The embodiments are not limited to operation within certain specific data processing environments, but can freely operate within multiple data processing environments. Additionally, although embodiments have been described using a specific series of transactions and steps, it will be apparent to those skilled in the art that the scope of this disclosure is not limited to the described series of transactions and steps. The various features and aspects of the above embodiments can be used individually or in combination.

[0404] Furthermore, while embodiments have been described using specific combinations of hardware and software, it should be recognized that other combinations of hardware and software are also within the scope of this disclosure. Embodiments may be implemented solely in hardware, solely in software, or using a combination thereof. The various processes described herein may be implemented on the same or different processors in any combination. Thus, where a component or module is described as being configured to perform certain operations, such a configuration may be implemented, for example, by designing electronic circuits to perform operations, by programming programmable electronic circuits (such as microprocessors), or any combination thereof. Processes may communicate using a variety of techniques, including but not limited to conventional techniques for inter-process communication, and different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0405] Therefore, the specification and drawings are to be considered illustrative rather than restrictive. However, it will be clear that additions, deletions, omissions, and other modifications and alterations may be made thereto without departing from the broader spirit and scope set forth in the claims. Thus, although specific disclosed embodiments have been described, they are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims.

[0406] In the context of describing the disclosed embodiments (particularly in the context of the following claims), the terms "a," "an," and "the," and similar pronouns, should be interpreted to cover both the singular and plural, unless otherwise stated herein or obviously contradicted by the context. Unless otherwise stated, the terms "comprising," "having," "owning," and "including" should be interpreted as open-ended terms (i.e., meaning "including, but not limited to"). The term "connected to" should be interpreted as partially or wholly included, attached to, or combined with, even if something else is involved. Unless otherwise indicated herein, the enumeration of numerical ranges herein is intended only as a concise way of expressing each separate value falling within that range, and each separate value is incorporated into the specification as if it were individually enumerated herein. Unless otherwise indicated herein or obviously contradicted by the context, all methods described herein may be performed in any suitable order. Unless otherwise claimed, the use of any and all examples or exemplary language (e.g., "such as") provided herein is intended only to better illustrate the embodiments and does not limit the scope of this disclosure. Nothing in the specification should be construed as indicating that any unclaimed element is essential to the practice of this disclosure.

[0407] Unless otherwise expressly stated, disjunctive language such as the phrase "at least one of X, Y, or Z" is intended to be understood in context as generally used to refer to items, terms, etc., which can be X, Y, or Z or any combination thereof (e.g., X, Y, and / or Z). Therefore, such disjunctive language is generally not intended and should not imply that certain embodiments require the presence of at least one of X, at least one of Y, or at least one of Z.

[0408] This document describes preferred embodiments of the present disclosure, including known best modes for carrying out the present disclosure. Variations of those preferred embodiments will become apparent to those skilled in the art after reading the foregoing description. Those skilled in the art should be able to appropriately employ such variations, and the present disclosure can be practiced in addition to those specifically described herein. Therefore, the present disclosure includes all modifications and equivalents of the subject matter recited in the appended claims as permitted by applicable law. Furthermore, unless otherwise indicated herein, any combination of the foregoing elements in all possible variations is included in this disclosure.

[0409] All references cited herein, including publications, patent applications and patents, are incorporated herein by reference to the extent that each reference is individually and specifically indicated as being incorporated and set forth herein by reference in its entirety.

[0410] In the foregoing description, aspects of this disclosure have been described with reference to specific embodiments thereof; however, those skilled in the art will recognize that this disclosure is not limited thereto. The various features and aspects of the foregoing disclosure may be used alone or in combination. Furthermore, embodiments may be utilized in any number of environments and applications beyond those described herein without departing from the broader spirit and scope of this specification. Therefore, the description and drawings are to be considered illustrative rather than restrictive.< / realm>

Claims

1. A computer-implemented method, comprising: The cloud defense system monitors reverse DNS traffic associated with the monitored environment, the reverse DNS traffic including a set of one or more reverse DNS resolver requests originating from the monitored environment, and a set of one or more responses generated by one or more DNS resolvers in response to the set of one or more reverse DNS resolver requests; The cloud defense system monitors one or more responses to the requests from the group of one or more reverse DNS resolvers; The cloud defense system collects and stores the raw data based on monitoring the reverse DNS traffic; The original data is enhanced by the cloud defense system to generate enhanced data; The cloud defense system uses the enhanced data to identify anomalous network activity associated with the monitored environment; as well as The cloud defense system outputs a signal indicating the abnormal network activity.

2. The computer-implemented method of claim 1, wherein identifying the anomalous network activity includes identifying a portion of the monitored environment experiencing the anomalous network activity, the portion of the monitored environment including one or more components of the monitored environment.

3. The method of claim 2, wherein the one or more components of the monitored environment include at least one of the following: a virtual cloud network (VCN) within the monitored environment, a region within the monitored environment, a group of one or more VCNs associated with a customer of a cloud service provider, a data center, virtual machines, and host machines within the monitored environment.

4. The computer-implemented method as described in any of the preceding claims, wherein identifying the anomalous network activity includes identifying the source of the anomalous network activity.

5. The computer-implemented method of claim 4, wherein the source is at least one of: (i) a part of the monitored environment or (ii) a component outside the monitored environment.

6. The computer-implemented method of claim 4 or claim 5, wherein identifying the source comprises performing the identification of at least one of the following: a first IP address associated with the source that triggered at least one of the one or more reverse DNS resolver requests in the set, a first fully qualified domain name (FQDN) associated with the first IP address, or an owner associated with the first FQDN.

7. The computer-implemented method as described in any of the preceding claims, further comprising: The cloud defense system initiates one or more actions in response to outputting a signal indicating the abnormal network activity.

8. The computer-implemented method of claim 7, wherein the set of one or more actions performs at least one of the following: (i) changing a set of rules associated with a component of the cloud defense system, (ii) isolating the system within the cloud service provider infrastructure (CSPI), and (iii) causing an alert to be generated.

9. The computer-implemented method of claim 8, wherein the alarm is a report and the alarm is sent to a user of the cloud defense system or a client of the CSPI.

10. The computer-implemented method as described in any of the preceding claims, wherein identifying the anomalous network activity comprises: A first baseline is generated using previously enhanced data, which was generated prior to the generation of the first baseline; Determine the deviation from the first baseline; as well as The deviation is identified as abnormal network activity.

11. The computer-implemented method of claim 10, wherein: The first baseline identifies a portion of the monitored environment and a first threshold associated with that portion of the monitored environment; and Determining the deviation includes determining, based on the enhanced data, that it has exceeded a first threshold associated with the portion.

12. The computer-implemented method of claim 11, wherein the first baseline represents the number of rDNS requests transmitted by the portion of the monitored environment within the set of one or more rDNS resolver requests.

13. The computer-implemented method of claim 11, wherein the first baseline represents the number of rDNS requests for resolving a set of one or more IP addresses transmitted by the portion of the monitored environment within the set of one or more rDNS resolver requests.

14. The computer-implemented method of claim 11, wherein the first baseline is different from the second baseline, the second baseline identifying a second portion of the monitored environment and a second threshold different from the first threshold.

15. The computer-implemented method as described in any of the preceding claims, wherein the set of one or more reverse DNS resolver requests is generated by one or more VCNs, one or more zones, or one or more virtual machines.

16. The computer-implemented method as described in any of the preceding claims, wherein the original data and an external data source are used when generating the enhanced data.

17. A cloud defense system, comprising: A traffic monitoring system, wherein the traffic monitoring system monitors reverse DNS traffic associated with a monitored environment to obtain raw data, the reverse DNS traffic including a set of one or more reverse DNS resolver requests originating from the monitored environment, and a set of one or more responses generated by one or more DNS resolvers in response to the set of one or more reverse DNS resolver requests; A data augmentation system that uses the original data to generate augmented data; as well as A data analysis system that uses the enhanced data to generate one or more alerts or reports, or to identify one or more patterns in the enhanced data.

18. The cloud defense system of claim 17, wherein the data analysis system identifies portions of the monitored environment experiencing abnormal network activity.

19. The cloud defense system of claim 17 or claim 18, wherein the traffic monitoring system monitors reverse DNS resolver requests generated by one or more VCNs, one or more regions, or one or more virtual machines.

20. A non-transitory computer-readable medium storing a set of instructions, which, when executed by one or more processors, cause processing to be performed, the processing comprising: The cloud defense system monitors reverse DNS traffic associated with the monitored environment, the reverse DNS traffic including a set of one or more reverse DNS resolver requests originating from the monitored environment, and a set of one or more responses generated by one or more DNS resolvers in response to the set of one or more reverse DNS resolver requests; The cloud defense system monitors one or more responses to the requests from the group of one or more reverse DNS resolvers; The cloud defense system collects and stores the raw data based on monitoring the reverse DNS traffic; The original data is enhanced by the cloud defense system to generate enhanced data; The cloud defense system uses the enhanced data to identify anomalous network activity associated with the monitored environment; as well as The cloud defense system outputs a signal indicating the abnormal network activity.

Images