DNS Recurrent PTR Signal Analysis

Abstract

This document describes cybersecurity techniques for monitoring cloud environments and identifying potential issues, including malicious threats, against the monitored cloud environment using actionable telemetry. It also describes techniques for monitoring and collecting data related to reverse or recursive DNS (rDNS) traffic associated with the monitored cloud environment. Recursive DNS traffic includes recursive DNS (rDNS) requests originating from the cloud environment and the responses received from DNS resolvers to those requests. This collected data is then analyzed to identify potential threats against the monitored cloud environment. The collected data may be analyzed to identify the source of potential threats and to identify one or more parts of the cloud environment that are targeted by the threats. This analysis may result in the generation of alerts, the taking of actions (e.g., protective measures), the generation of reports, the recognition of patterns, etc.

Description

Technical Field

[0001] Cross-Reference to Related Applications This application is a PCT application claiming priority to U.S. Patent Application No. 18 / 367,938 filed on September 13, 2023, the entire contents of which are incorporated herein by reference for all purposes.

[0002] Field The present disclosure generally relates to cybersecurity technology, and more specifically, to DNS recursive PTR signals analysis.

Background Art

[0003] Background Adoption of cloud services has increased dramatically in recent years. As a result, there is a growing number of cloud services providers (CSPs) that provide customers with subscriptions to one or more cloud services. In a typical scenario, a CSP provides a cloud environment with infrastructure provided by the CSP, and the cloud environment is used to provide one or more services that the CSP delivers to customers. The cloud environment may include networked computing resources, memory resources, network resources, software resources, and other types of resources that are used for provisioning cloud services. A cloud environment typically includes a physical network layer (referred to as a substrate layer) on which one or more virtual networks are supported and used to provide cloud services.

Summary of the Invention

Problem to be Solved by the Invention

[0004] Due to their decentralized nature and complexity, cloud environments provided by CSPs are highly vulnerable to malicious cyberattacks. For CSPs, being able to protect their cloud environments from cyberattacks caused by malicious actors is critical. This is essential for protecting customer data and other customer resources entrusted to the CSP. The negative reputation resulting from a security breach can devastate a CSP's business. Therefore, CSPs are constantly seeking new and innovative ways to better protect their cloud environments. [Means for solving the problem]

[0005] overview This disclosure relates to cybersecurity technologies, more specifically to technologies for monitoring cloud environments and identifying potential issues, including malicious threats, against the monitored cloud environments using manipulable telemetry. One implementation describes a technology for monitoring and collecting data related to reverse or recursive DNS (rDNS) traffic associated with a monitored cloud environment. Recursive DNS traffic includes recursive DNS (rDNS) requests originating from the cloud environment and the responses received from DNS resolvers to those requests. This collected data is then analyzed to identify potential threats against the monitored cloud environment. The collected data may be analyzed to identify the source of the potential threat and to identify one or more parts of the cloud environment that are targeted by the threat.

[0006] This disclosure relates to monitoring DNS recursive resolver traffic, more specifically, PTR record resolution. Such monitoring may track the regions, VCNs, and / or host machines associated with each rDNS request to determine the number of regions, VCNs, and / or host machines that attempted to resolve each PTR record. After the data has been collected, observations may be aggregated, analyzed, and / or stored for subsequent use. Analysis of rDNS request and response data may include determining which systems are being targeted by malicious activity (e.g., malicious actors, anomalous activity), and / or determining what and / or who is causing the irregular activity. As a result of the analysis, alerts may be generated, actions (e.g., protective measures) may be taken, reports may be generated, patterns may be recognized, etc. Various embodiments are described herein, including methods, systems, non-temporary computer-readable storage media for storing programs, code, or instructions executable by one or more processors. Some embodiments may be implemented by using a computer program product that includes computer programs / instructions, which, when executed by a processor, cause the processor to perform one of the methods described herein.

[0007] In some implementations, a technique (e.g., method, system, computer-readable medium) includes monitoring reverse DNS traffic associated with a monitored environment by a cloud defense system, where the reverse DNS traffic includes one or more sets of reverse DNS resolver requests originating from the monitored environment, and one or more sets of responses generated by one or more DNS resolvers in response to one or more sets of reverse DNS resolver requests. These techniques may further include monitoring one or more responses to one or more sets of reverse DNS resolver requests by the cloud defense system. These techniques may further include collecting and storing raw data based on the monitoring of reverse DNS traffic by the cloud defense system, augmenting the raw data to generate augmented data by the cloud defense system, and using the augmented data to identify irregular network activity associated with the monitored environment by the cloud defense system. These techniques may further include outputting signals indicating irregular network activity by the cloud defense system.

[0008] In one implementation, identifying irregular network activity involves identifying the portion of the monitored environment where the irregular network activity is occurring, and this portion of the monitored environment comprises one or more components of the monitored environment. In one implementation, one or more components of the monitored environment include at least one of the following: a virtual cloud network (VCN) within the monitored environment, a region within the monitored environment, a set of one or more VCNs associated with a cloud service provider's customer, a data center within the monitored environment, a virtual machine, and a host machine.

[0009] In some implementations, identifying irregular network activity involves identifying the source of that irregular network activity. In some implementations, this source is part of the monitored environment. In other implementations, this source is a component outside the monitored environment.

[0010] In some implementations, identifying this source involves identifying at least one of the following: a first IP address associated with the source that caused at least one reverse DNS request in a set of one or more reverse DNS resolver requests; a first fully qualified domain name (FQDN) associated with the first IP address; or an owner associated with the first FQDN.

[0011] In some implementations, these techniques further include the cloud defense system initiating a set of one or more actions in response to the output of a signal indicating irregular network activity. In some implementations, the set of one or more actions perform at least one of the following: (i) modifying a set of rules associated with a component of the cloud defense system; (ii) isolating the system within the cloud server provider infrastructure (CSPI); and (iii) causing an alert to be generated. In some implementations, the alert is a report, which is sent to a user of the cloud defense system or a customer of the CSPI.

[0012] In one implementation, identifying irregular network activity involves generating a first baseline using prior augmented data, which is generated before generating the first baseline, determining deviations from the first baseline, and identifying these deviations as irregular network activity. In another implementation, the first baseline identifies a portion of the monitored environment and a first threshold associated with that portion, and determining deviations involves determining, based on the augmented data, that the first threshold associated with this portion has been exceeded. In yet another implementation, the first baseline represents the number of rDNS requests in a set of one or more rDNS resolver requests sent by a portion of the monitored environment. In yet another implementation, the first baseline represents the number of rDNS requests in a set of one or more rDNS resolver requests sent by a portion of the monitored environment to resolve a set of one or more IP addresses. In yet another implementation, the first baseline is different from a second baseline which identifies a second portion of the monitored environment using a second threshold different from the first threshold.

[0013] In one implementation, a set of one or more reverse DNS resolver requests is generated by one or more VCNs, one or more regions, or one or more virtual machines.

[0014] In some implementations, raw data and external data sources are used when generating augmented data.

[0015] The foregoing, along with other features and embodiments, will become even clearer when referring to the following specification, claims, and accompanying drawings. [Brief explanation of the drawing]

[0016] [Figure 1]This is a high-level diagram of a distributed environment showing a virtual cloud network or overlay cloud network hosted by a cloud service provider infrastructure, according to one embodiment. [Figure 2] This figure shows a simplified architectural diagram of the physical components in the physical network within the CSPI according to one embodiment. [Figure 3] This figure shows an exemplary configuration within a CSPI in which a host machine is connected to multiple network virtualization devices (NVDs), according to one embodiment. [Figure 4] This figure shows the connection between a host machine and an NVD to implement I / O virtualization to support multi-tenancy functionality, according to one embodiment. [Figure 5] This figure shows a simplified block diagram of a physical network provided by CSPI according to one embodiment. [Figure 6] This figure shows a simplified flowchart of the systems and processes involved in rDNS requests and rDNS responses according to some embodiments. [Figure 7] This figure shows a simplified flowchart of the systems and processes involved in obtaining monitored traffic data from rDNS requests and rDNS responses, according to some embodiments. [Figure 8] This figure shows a simplified architectural diagram of a system that communicates with and constitutes a cloud defense system, according to one embodiment. [Figure 9] This figure shows a simplified flowchart 900 illustrating a method for protecting a monitored environment using rDNS traffic, according to some embodiments. [Figure 10] This figure shows a simplified flowchart for monitoring a monitored environment, determining baseline network activity, and deciding on actions to take when irregular activity is identified, according to some embodiments. [Figure 11] FIG. 2 is a simplified flow diagram for determining a baseline rDNS request volume of a monitored environment, in accordance with some embodiments. [Figure 12] FIG. 3 is a simplified flow diagram for detecting anomalous network behavior in a monitored environment, in accordance with some embodiments. [Figure 13A] FIG. 4 is a simplified flow diagram for detecting malicious actors using a cloud defense system, in accordance with some embodiments. [Figure 13B] FIG. 5 is a simplified flow diagram for detecting malicious actors using a cloud defense system, in accordance with some embodiments. [Figure 14] FIG. 6 is a block diagram illustrating one pattern for implementing cloud infrastructure as a service system, in accordance with at least one embodiment. [Figure 15] FIG. 7 is a block diagram illustrating another pattern for implementing cloud infrastructure as a service system, in accordance with at least one embodiment. [Figure 16] FIG. 8 is a block diagram illustrating another pattern for implementing cloud infrastructure as a service system, in accordance with at least one embodiment. [Figure 17] FIG. 9 is a block diagram illustrating another pattern for implementing cloud infrastructure as a service system, in accordance with at least one embodiment. [Figure 18] FIG. 10 is a block diagram illustrating an exemplary computer system, in accordance with at least one embodiment. Description of Embodiments

[0017] DETAILED DESCRIPTION In the following description, certain details are provided to allow for a full understanding of certain embodiments for illustrative purposes. However, it is clear that various embodiments can be practiced without those specific details. Each figure and description is not intended to be limiting. The word “exemplary” is used herein to mean “serving as an example, case, or illustration.” Not all embodiments or designs described herein as “exemplary” should be construed as necessarily preferable or advantageous over other embodiments or designs.

[0018] This disclosure relates to cybersecurity technologies, more specifically to technologies for monitoring cloud environments and identifying potential issues, including malicious threats, against the monitored cloud environments using manipulable telemetry. One implementation describes a technology for monitoring and collecting data related to reverse lookup or recursive DNS (rDNS) traffic associated with a monitored cloud environment. Recursive DNS traffic includes recursive DNS (rDNS) requests originating from the cloud environment and the responses received from DNS resolvers to those requests. This collected data is then analyzed to identify potential threats against the monitored cloud environment. The collected data may be analyzed to identify the source of the potential threat and to identify one or more parts of the cloud environment that are targeted by the threat.

[0019] In one implementation, a cloud defense system is provided, which is configured to monitor and collect data related to rDNS requests originating from the cloud environment and the corresponding responses generated by the DNS resolver. The cloud defense system is configured to generate augmented data by augmenting the collected raw data with additional data obtained from one or more additional data sources. The cloud defense system then uses the augmented data to identify portions of the monitored cloud environment that are exhibiting anomalous behavior and are therefore potentially targets for malicious attacks. In one implementation, the cloud defense system uses the augmented data to generate baselines of the monitored cloud environment over a period of time. In one implementation, multiple different baselines may be generated for different portions of the monitored cloud environment. The cloud defense system then identifies deviations from the baselines and identifies the portions of the monitored cloud environment where the deviations are occurring as potential targets for malicious attacks.

[0020] In some implementations, the cloud defense system uses augmented data to also identify the source of threats against the monitored cloud environment. For example, a certain IP address causing an unusually large amount of rDNS traffic may be identified as a potential malicious actor. The source of the threat may be located outside or inside the monitored cloud environment. In some implementations, multiple different baselines may be generated for sources sending traffic to the monitored cloud environment. The cloud defense system may then identify deviations from such baselines and identify the source associated with the deviation as a potential source of a malicious attack.

[0021] Cloud defense systems reduce false positives of threats. For example, various normal activities can trigger rDNS requests. An example is a scan performed by a source. In some cases, the scan may be performed by a legitimate source (e.g., a scan performed by a trusted system, e.g., an internal network security team, to identify system vulnerabilities), while in some other cases, the scan may be performed by malware attempting to propagate to multiple host machines or servers and / or virtual machines. By using baselines generated over a period of time and using those baselines to find deviations, the cloud defense systems described herein can distinguish between other normal activities and activities that are actually threats. In this way, cloud defense systems reduce the occurrence of false positives.

[0022] The cloud defense system collects data related to rDNS traffic in the monitored environment and further augments the collected data using additional data sources. This augmented data provides a situational view of the state of the monitored environment, facilitating a more accurate identification of parts of the monitored environment that may be under attack, and / or sources that may be maliciously attacking at least parts of the monitored environment. Based on this context, the cloud defense system can identify the intent behind data communications with the monitored environment and distinguish malicious activity from normal baseline activity. This improves the overall efficiency and usefulness of the cloud defense system. In addition to efficiency improvements through reducing false positives, efficiency can also be improved by removing noise from network traffic data before it is processed (filtering network traffic data), thereby reducing the time and resources required to process filtered network traffic data compared to unfiltered network traffic data.

[0023] When anomalous activity is identified, and a portion of the monitored cloud environment is identified as experiencing anomalous behavior, or a specific source is identified as the source of anomalous behavior occurring in a portion of the monitored cloud environment, the cloud defense system may trigger one or more actions. For example, one or more actions may be initiated to isolate the portion of the cloud environment experiencing anomalous behavior, contain the anomalous activity within a portion of the cloud environment, or remediate a cyber breach or threat. These actions may include, for example, configuring a firewall, taking host machines offline, configuring a list of IP addresses or FQDNs to be blocked, or isolating a specific VCN or set of VCNs.

[0024] The cloud defense system is configured to collect and store data related to rDNS traffic at different levels of the hierarchy of the monitored cloud environment. For example, in one embodiment, the cloud defense system collects rDNS traffic data at the level of each virtual cloud network (VCN), where a VCN may include multiple host machines running on one or more host machines. The cloud defense system monitors rDNS requests originating from VCNs within the monitored cloud environment and the corresponding DNS resolver responses. Data collected at the VCN level can be used to identify irregular behavior at the VCN level. For VCNs within a data center, the data collected for the VCN can be aggregated to represent data for the data center. Aggregated data at the data center level can be used to identify irregular network activity at the data center level. For one or more data centers within a region, the collected data center data can be aggregated to represent data for the region. Aggregated data at the region level can be used to identify irregular network activity at the region level. For a global area containing one or more regions, the data collected for the region can be aggregated to represent data for the global area. Aggregated data at the global area level can be used to identify irregular network activity at the global area level. In this way, the cloud defense system can collect, aggregate, and analyze rDNS traffic data collected across different hierarchical architecture levels of the monitored cloud environment. The collected data can also be used to monitor irregular activity at the per-customer level, and customers may be associated with one or more VCNs.

[0025] In one embodiment, the cloud defense system may include multiple systems, including a traffic monitoring system, a data augmentation system, a data analysis system, a report generator system, and a query system. The traffic monitoring system may monitor and collect data related to rDNS traffic associated with the monitored cloud environment. The monitored rDNS traffic may include rDNS requests originating from the monitored cloud environment and corresponding responses received from one or more DNS resolvers. Monitoring and collection may be performed at various hierarchical levels of the cloud environment, including VCNs, customers, data centers, regions, global areas, etc.

[0026] In one implementation, the data augmentation system is responsible for generating augmented data by augmenting the raw data collected by the traffic monitoring system. Various methods exist for how the raw data can be augmented to generate the augmented data. In one use case, the data augmentation system may augment the raw data using data obtained from one or more data sources, including external third-party data sources.

[0027] In one implementation, the data analysis system is responsible for analyzing augmented data and outputting analysis results. The data may be analyzed according to different dimensions and parameters. For example, the data analysis system may analyze augmented data to determine baseline information for a monitored cloud environment. For example, the baseline may identify the number of rDNS requests originating from the monitored cloud environment for a given IP address under normal operating conditions. The baseline information may include information related to one or more parameters, such as the number of rDNS requests and / or responses, related to a specific IP, related to a specific fully qualified domain name (FQDN), related to a specific owner of one or more FQDNs, etc. The baseline information may identify one or more thresholds related to these different parameters. The data analysis system can then identify and track deviations from the baseline behavior. If a deviation from the baseline behavior exceeds a pre-established threshold, the deviation may be flagged as irregular behavior by the data analysis system. The data analysis system may identify the part of the monitored cloud environment where the irregular behavior is occurring (e.g., a specific VCN, a group of VCNs, a specific customer, a data center, a region, etc.). The data analysis system may also identify sources (e.g., specific IP addresses, FQDNs, entities, etc.) that cause or trigger abnormal behavior. In one embodiment, the data analysis system can use augmented data to identify and evaluate whether one or more patterns exist in the collected data and augmented data. Deviations from normal patterns may be identified as abnormal, potentially malicious behavior.

[0028] In some embodiments, upon detecting abnormal behavior, the data analysis system may transmit a signal to a downstream system, which then operates according to the signal received from the data analysis system. For example, the data analysis system may transmit a signal indicating abnormal behavior to a downstream warning system, which may generate one or more warnings accordingly. These warnings may be associated with different severity levels based on the information contained in the signal received from the data analysis system. In one implementation, the warning system may be configured using rules and / or machine learning models, which it uses to determine which warnings to generate, when the warnings are generated, the content of the warnings, the recipients of the warnings, the communication channels used to deliver the warnings to the intended recipients, and so on.

[0029] As another example, a data analysis system may send signals indicating abnormal behavior to a downstream action system, which may then initiate one or more actions. These actions may include actions to mitigate or isolate the target or source of the abnormal behavior, preventive measures, corrective measures, etc. In one implementation, the action system may be configured using rules and / or machine learning models, which the action system uses to determine which actions to initiate, when to initiate them, the target of the actions (e.g., which VCN, host machine, etc.), etc. Examples of actions include isolating a VCN from interaction with other systems in the monitored cloud environment, disconnecting a specific host machine from the network, configuring a firewall, and other actions.

[0030] As yet another example, a data analysis system may send signals indicating abnormal behavior to a report generation system, which may then generate one or more reports and send these reports to pre-configured recipients. In one implementation, the report generator system may be configured using rules and / or machine learning models, which the report generator system uses to determine which reports to generate, when the reports are generated, the content of the reports, the recipients of the reports, the communication channels used to deliver the reports to the intended recipients, and so on.

[0031] In one embodiment, the cloud defense system may provide a query system that enables users of the cloud defense system to query raw and / or augmented data and perform their own analysis of the data. The query system can support various types of queries, such as SQL queries and natural language queries. Examples of queries include queries to identify all instances of reverse DNS requests generated for a specific IP in a region over a specific period, the number of rDNS requests originating from a specific VCN over a specific period, and all FQDNs involved in rDNS requests exceeding a certain threshold over a specific period. The query system enables users of the cloud defense system to retrieve relevant information from the collected and augmented data, perform their own analysis, and so on.

[0032] The embodiments described herein can leverage rDNS request and response analysis to classify network activity within a monitored cloud environment and perform processes that may lead to a better understanding of the root cause of incidents and the intentions surrounding such incidents.

[0033] This disclosure describes a novel solution for protecting cloud environments from malicious attacks using technology to monitor, collect, and analyze data related to rDNS traffic associated with cloud environments. Because the data is collected at the VCN level, the identity of host machines operating on the VCN, which may be associated with cloud service customers, is not made available to the cloud defense system. This is important for many cloud service customers who prefer that information about their payloads remain private.

[0034] The various solutions described in this disclosure provide novel methods for protecting cloud environments using data collected from monitoring rDNS requests and / or responses. Some embodiments can reduce the processing time required to analyze network activity and improve the accuracy of detecting irregular network activity (e.g., malicious network activity, anomalous network activity). Recognition of irregular network activity can extend the operational time of the network and its systems and may also enhance data protection from threats. In some embodiments, certain network traffic may be blocked as a result of actions taken by the cloud defense system, thus enhancing connectivity to the cloud service provider infrastructure systems.

[0035] Furthermore, irregular activity identified using the scale of the cloud defense system can be relayed to cloud infrastructure customers and users so that they can receive similar benefits.

[0036] Exemplary virtual network architecture The term "cloud service" is typically used to refer to services made available on demand (e.g., through a subscription model) by users or customers using systems and infrastructure provided by a Cloud Service Provider (CSP) (cloud infrastructure). Typically, the servers and systems that make up the CSP's infrastructure are separate from the customer's own on-premises servers and systems. Therefore, customers can utilize cloud services provided by the CSP without needing to purchase separate hardware and software resources for the service. Cloud services are designed to provide subscribers with easy and scalable access to application and computing resources without requiring the customer to invest in procuring the infrastructure used to deliver the service.

[0037] There are multiple cloud service providers offering various types of cloud services. These include various types or models of cloud services, such as SaaS (Software-as-a-Service), PaaS (Platform-as-a-Service), and IaaS (Infrastructure-as-a-Service).

[0038] Customers can subscribe to one or more cloud services provided by a CSP. Customers can be any entity, such as an individual, organization, or company. When a customer subscribes to or registers for a service provided by a CSP, a tenancy or account is created for that customer. The customer can then use that account to access one or more subscribed cloud resources associated with the account.

[0039] As mentioned earlier, IaaS (Infrastructure as a Service) is a specific type of cloud computing service. In the IaaS model, the CSP provides the infrastructure (called Cloud Service Provider Infrastructure or CSPI) that can be used by the customer to build their own customizable network and deploy their resources. Thus, the customer's resources and network are hosted in a distributed environment by the infrastructure provided by the CSP. This differs from traditional computing, where the customer's resources and network are hosted by infrastructure provided by the customer.

[0040] A CSPI can have interconnected, high-performance computing resources, including various host machines, memory resources, and network resources that form a physical network, also known as an underlay network. Resources in a CSPI can be distributed across one or more data centers that can be geographically distributed across one or more geographical areas. To provide a virtualized distributed environment, virtualization software may run on these physical resources. This virtualization creates an overlay network (also known as a software-based network, software-defined network, or virtual network) on top of the physical network. The physical network of a CSPI provides the foundation for creating one or more overlay or virtual networks on top of the physical network. The physical network (or underlay network) includes physical network devices such as physical switches, routers, computers, and host machines. An overlay network is a logical (or virtual) network that operates on top of the physical underlay network. A particular physical network can support one or more overlay networks. Overlay networks typically use encapsulation techniques to distinguish traffic belonging to different overlay networks. A virtual network or overlay network is also called a virtual cloud network (VCN). Virtual networks are implemented using software virtualization technologies (e.g., hypervisors, network virtualization devices (NVDs) (e.g., smart NICs), top-of-rack (TOR) switches, virtualization functions implemented by smart TORs that perform one or more functions performed by NVDs, and other mechanisms) to create a layer of network abstraction that can run on top of a physical network. Virtual networks can take many forms, including peer-to-peer networks and IP networks.A virtual network is typically either a Layer 3 IP network or a Layer 2 VLAN. This method of virtual networking or overlay networking is often referred to as a virtual Layer 3 network or overlay Layer 3 network. Examples of protocols developed for virtual networks include IP-in-IP (or Generic Routing Encapsulation (GRE)), Virtual Extensible LAN (VXLAN - IETF RFC 7348), Virtual Private Networks (VPN) (e.g., MPLS Layer 3 Virtual Private Network (RFC 4364)), VMware's NSX, and Geneve (Generic Network Virtualization Encapsulation).

[0041] In the case of IaaS, the infrastructure provided by the CSP (CSPI) can be configured to deliver virtualized computing resources over a public network (e.g., the internet). In the IaaS model, the cloud computing service provider can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, the IaaS provider may also provide various services (e.g., billing, monitoring, logging, security, load balancing, and clustering, etc.) as they arise in conjunction with those infrastructure components. Therefore, since these services can be policy-driven, IaaS users may be able to implement policies to drive load balancing and maintain application availability and performance. The CSPI provides infrastructure and a set of complementary cloud services that enable customers to build and run a wide range of applications and services in a highly available hosted distributed environment. The CSPI provides high-performance computing resources and computing power, as well as storage capacity, within a flexible virtual network that can be securely accessed from various networked locations, such as the customer's on-premises network. When a customer subscribes to or registers for an IaaS service provided by a CSP, the tenancy created for that customer is a secure and isolated partition within the CSP where the customer can create, organize, and manage their own cloud resources.

[0042] Customers can build their own virtual networks using the computing, memory, and network resources provided by CSPI. One or more customer resources or workloads, such as compute instances, can be deployed to these virtual networks. For example, a customer can build one or more customizable private virtual networks called Virtual Cloud Networks (VCNs) using the resources provided by CSPI. A customer can deploy one or more customer resources, such as compute instances, to their VCNs. Compute instances can take the form of virtual machines, bare metal instances, etc. Thus, CSPI provides the infrastructure and a set of complementary cloud services that enable customers to build and run a wide range of applications and services in a highly available hosted virtual environment. Customers do not manage or control the underlying physical resources provided by CSPI, but they can control the operating system, storage, and deployed applications, and, in some cases, have limited control over selected network components (e.g., firewalls).

[0043] A CSP may provide a console that enables customers and network administrators to configure, access, and manage resources deployed in the cloud using CSPI resources. In one embodiment, this console provides a web-based user interface that can be used to access and manage CSPI. In some implementations, the console is a web-based application provided by the CSP.

[0044] CSPI can support single-tenancy or multi-tenancy architectures. In a single-tenancy architecture, software components (e.g., applications, databases) or hardware components (e.g., host machines or servers) work for a single customer or tenant. In a multi-tenancy architecture, software components or hardware components work for multiple customers or tenants. Therefore, in a multi-tenancy architecture, CSPI resources are shared among multiple customers or tenants. In a multi-tenancy situation, precautions are taken and preventative measures are introduced within CSPI to ensure that each tenant's data is isolated and remains invisible to other tenants.

[0045] In a physical network, a network endpoint ("endpoint") refers to a computing device or system that is connected to a physical network and communicates with the connected network. Network endpoints within a physical network may be connected to a local area network (LAN), a wide area network (WAN), or other types of physical networks. Examples of traditional endpoints within a physical network include modems, hubs, bridges, switches, routers, and other network devices, as well as physical computers (or host machines). Each physical device within a physical network has a fixed network address that can be used to communicate with the device. This fixed network address may be a Layer 2 address (e.g., a MAC address), a fixed Layer 3 address (e.g., an IP address), etc. In a virtual environment or virtual network, endpoints can include various virtual endpoints, such as virtual machines hosted by components of a physical network (e.g., hosted by a physical host machine). These endpoints within a virtual network are addressed by overlay addresses, such as overlay Layer 2 addresses (e.g., overlay MAC addresses) and overlay Layer 3 addresses (e.g., overlay IP addresses). Network overlays enable flexibility by allowing network administrators to move overlay addresses between network endpoints using software management (e.g., by software implementing the control plane of the virtual network). Therefore, unlike physical networks, in virtual networks, overlay addresses (e.g., overlay IP addresses) can be moved from one endpoint to another using network management software. Because virtual networks are built on top of physical networks, communication between components within a virtual network involves both the virtual network and the underlying physical network.To facilitate such communication, CSPI components are configured to learn and store mappings that map overlay addresses in the virtual network to actual physical addresses in the underlying network, and vice versa. These mappings are then used to facilitate communication. Customer traffic is encapsulated to facilitate routing within the virtual network.

[0046] Therefore, physical addresses (e.g., physical IP addresses) are associated with components within a physical network, while overlay addresses (e.g., overlay IP addresses) are associated with entities within a virtual network or overlay network. A physical IP address is an IP address associated with a physical device (e.g., a network device) within an underlying network or physical network. For example, each NVD has an associated physical IP address. An overlay IP address is an overlay address associated with an entity within an overlay network, such as being associated with a compute instance within a customer's virtual cloud network (VCN). Two different customers or tenants, each having their own private VCN, may be able to use the same overlay IP address within their VCNs without knowing anything about each other. Both physical IP addresses and overlay IP addresses are types of real IP addresses. These IP addresses exist separately from virtual IP addresses. A virtual IP address is typically a single IP address that represents or maps to multiple real IP addresses. A virtual IP address provides a one-to-many mapping between a virtual IP address and multiple real IP addresses. For example, a load balancer can use a VIP to map to or represent multiple servers, each server having its own actual IP address.

[0047] The cloud infrastructure, or CSPI, is physically hosted in one or more data centers in one or more regions worldwide. The CSPI may include components within a physical network or underpinning network, and virtual components (e.g., virtual networks, compute instances, virtual machines, etc.) within a virtual network built on top of the physical network components. In some embodiments, the CSPI is organized and hosted within realms, regions, and availability domains. A region is typically a localized geographical area containing one or more data centers. Regions are generally independent of each other and may be separated by vast distances, for example, across multiple countries or continents. For example, one region might be in Australia, another in Japan, and yet another in India. CSPI resources are partitioned across regions such that each region contains an independent subset of CSPI resources within that region itself. Each region may provide a set of core infrastructure services and resources, including compute resources (e.g., bare metal servers, virtual machines, containers, and associated infrastructure), storage resources (e.g., block volume storage, file storage, object storage, archive storage), network resources (e.g., virtual cloud networks (VCNs), load balancing resources, connectivity to on-premises networks), database resources, edge network resources (e.g., DNS), and access management and monitoring resources. Each region generally has multiple paths to connect to other regions within the realm.

[0048] Because using nearby resources is faster than using distant resources, applications are generally deployed within the region where they are most frequently used (i.e., on the infrastructure associated with that region). Applications may also be deployed across different regions for various reasons, such as redundancy to mitigate the risk of region-wide events like major weather or earthquakes, to meet changing requirements regarding legal jurisdictions, tax areas, and other business or social standards.

[0049] Data centers within a region can be further organized and subdivided into availability domains (ADs). An availability domain may correspond to one or more data centers located within a region. A region may consist of one or more availability domains. In such a distributed environment, CSPI resources are either region-specific, such as virtual cloud networks (VCNs), or availability domain-specific, such as compute instances.

[0050] Active Directory (AD) instances within a region are configured to be isolated from each other, fault-tolerant, and extremely unlikely to fail simultaneously. This is achieved by ADs that do not share critical infrastructure resources such as networks, physical cables, cable routes, and cable entry points, so that a failure in one AD within a region is unlikely to affect the availability of other ADs within the same region. ADs within the same region may be connected to each other by low-latency, high-bandwidth networks, enabling the provision of high-availability connectivity to other networks (e.g., the internet, customer on-premises networks, etc.) and the construction of replicated systems across multiple ADs for both high availability and disaster recovery. Cloud services use multiple ADs to ensure high availability and protect against resource failures. As the infrastructure provided by IaaS providers grows, more regions and ADs with additional capacity may be added. Traffic between availability domains is typically encrypted.

[0051] In one embodiment, regions are grouped into realms. A realm is a logical collection of regions. Realms are isolated from each other and do not share any data. Regions within the same realm can communicate with each other, but regions in different realms cannot. A customer's tenancy or account, along with the CSP, resides within a single realm and may be distributed across one or more regions belonging to that realm. Typically, when a customer subscribes to an IaaS service, a tenancy or account for that customer is created within a region specified by the customer within a realm (referred to as the "home" region). A customer can extend their tenancy across one or more other regions within a realm. A customer cannot access regions that are not in the realm where their tenancy resides.

[0052] An IaaS provider can offer multiple realms, each catering to the specific needs of a particular set of customers or users. For example, a commercial realm might be offered to commercial customers. Another example is a realm that is offered to a specific country for customers within that country. Yet another example is a government realm that is offered to governments, etc. For example, a government realm might be tailored to the specific needs of a particular government and may have a higher level of security than a commercial realm. For example, Oracle Cloud Infrastructure (OCI) currently offers two realms: one for commercial regions and another for government cloud regions (e.g., FedRAMP certified and IL5 certified).

[0053] In some embodiments, an Active Directory (AD) may be subdivided into one or more fault domains. A fault domain is a group of infrastructure resources within the AD to provide anti-affinity. Fault domains enable the distribution of compute instances so that multiple compute instances do not reside on the same physical hardware within a single AD. This distribution is known as anti-affinity. A fault domain refers to a set of hardware components (computers, switches, etc.) that share a single point of failure. The compute pool is logically divided into fault domains. Therefore, a hardware failure or maintenance event on compute hardware affecting one fault domain does not affect instances in other fault domains. Depending on the embodiment, the number of fault domains per AD may vary. For example, in some embodiments, each AD contains three fault domains. Fault domains function as logical data centers within the AD.

[0054] When a customer subscribes to an IaaS service, resources from CSPI are provisioned for the customer and associated with the customer's tenancy. The customer can use these provisioned resources to build private networks and deploy resources to these networks. A customer's network hosted in the cloud by CSPI is called a Virtual Cloud Network (VCN). A customer can configure one or more Virtual Cloud Networks (VCNs) using the CSPI resources allocated to them. A VCN is a Virtual Private Network or Software-Defined Private Network. Customer resources deployed within a customer's VCN can include compute instances (e.g., virtual machines, bare metal instances) and other resources. These compute instances may represent various customer workloads, such as applications, load balancers, and databases. Compute instances deployed in a VCN can communicate with publicly accessible endpoints ("public endpoints") via public networks such as the internet, with other instances within the same VCN or other VCNs (e.g., other VCNs of the customer, or VCNs not belonging to the customer), with the customer's on-premises data centers or networks, and with service endpoints and other types of endpoints.

[0055] A CSP can provide a variety of services using a CSPI. In some cases, a CSPI customer can act like a service provider themselves and provide services using CSPI resources. A service provider can expose service endpoints characterized by identifying information (e.g., IP address, DNS name, and DNS port). A customer's resources (e.g., compute instances) can consume a particular service by accessing the service endpoint exposed by the service for that particular service. These service endpoints are generally publicly accessible by users using the public IP address associated with the endpoint over a public communication network such as the internet. Publicly accessible network endpoints are sometimes called public endpoints.

[0056] In one embodiment, a service provider may expose a service through an endpoint for the service (sometimes called a service endpoint). Customers of the service can then access the service using this service endpoint. In one implementation, the service endpoint provided for a service may be accessible to multiple customers who wish to consume that service. In another implementation, a dedicated service endpoint may be provided to a customer, allowing only that customer to access the service using that dedicated service endpoint.

[0057] In one embodiment, when a VCN is created, it is associated with a private overlay classless inter-domain routing (CIDR) address space, which is a range of private overlay IP addresses assigned to the VCN (e.g., 10.0 / 16). The VCN includes associated subnets, route tables, and gateways. A VCN resides within a single region but can span one, more, or all of the region's availability domains. A gateway is a virtual interface configured for a VCN that enables communication of traffic to and from the VCN and to one or more endpoints outside the VCN. One or more different types of gateways may be configured for a VCN to enable communication with different types of endpoints.

[0058] A VCN can be subdivided into one or more subnets, such as one or more subnets. Therefore, a subnet is a unit of configuration or subdivision that can be created within a VCN. A VCN can contain one or more subnets. Each subnet within a VCN is associated with a contiguous range of overlay IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24) that represents a subset of the address space within the VCN's address space and does not overlap with other subnets within that VCN.

[0059] Each compute instance is associated with a virtual network interface card (VNIC) that allows the compute instance to join a subnet in the VCN. A VNIC is a logical representation of a physical network interface card (NIC). Generally, a VNIC is the interface between an entity (e.g., compute instance, service) and a virtual network. A VNIC resides in a subnet and has one or more associated IP addresses and associated security rules or security policies. A VNIC is equivalent to a Layer 2 port on a switch. A VNIC is connected to a compute instance and connects to a subnet in the VCN. The VNIC associated with a compute instance allows the compute instance to become part of a subnet in the VCN and enables the compute instance to communicate (e.g., send and receive packets) with endpoints on the same subnet as the compute instance, endpoints in different subnets within the VCN, or endpoints outside the VCN. Thus, the VNIC associated with a compute instance determines how the compute instance connects to endpoints inside and outside the VCN. A VNIC for a compute instance is created and associated with that compute instance when the compute instance is created and added to a subnet in the VCN. For a subnet containing a set of compute instances, the subnet contains VNICs corresponding to the set of compute instances, and each VNIC connects to one of the compute instances in the set.

[0060] Each compute instance is assigned a private overlay IP address via the VNIC associated with it. This private overlay IP address is assigned to the VNIC associated with the compute instance when the compute instance is created and is used to route traffic to and from the compute instance. All VNICs within a given subnet use the same route table, security lists, and DHCP options. As mentioned earlier, each subnet within a VCN is associated with a contiguous range of overlay IP addresses (e.g., 10.0.0.0 / 24 and 10.0.1.0 / 24) that represents a subset of the address space within the VCN's address space and does not overlap with other subnets within that VCN. For a VNIC on a particular subnet of a VCN, the private overlay IP address assigned to the VNIC is an address from the contiguous range of overlay IP addresses assigned to that subnet.

[0061] In one embodiment, a compute instance may optionally be assigned additional overlay IP addresses in addition to its private overlay IP address, such as one or more public IP addresses if it is in a public subnet. These multiple addresses may be assigned to the same VNIC or across multiple VNICs associated with the compute instance. However, each instance has a primary VNIC, which is created during instance startup and associated with the overlay private IP address assigned to the instance, and this primary VNIC cannot be removed. Additional VNICs, called secondary VNICs, may be added to an existing instance in the same availability domain as the primary VNIC. All VNICs are in the same availability domain as the instance. Secondary VNICs may be in a subnet within the same VCN as the primary VNIC, or in a different subnet, either within the same VCN or a different VCN.

[0062] If a compute instance is located in a public subnet, it may optionally be assigned a public IP address. When a subnet is created, it can be designated as either a public or private subnet. A private subnet means that resources within the subnet (e.g., compute instances) and associated VNICs cannot have public overlay IP addresses. A public subnet means that resources within the subnet and associated VNICs can have public IP addresses. Customers can specify subnets to reside within a single availability domain or across multiple availability domains within a region or realm.

[0063] As mentioned earlier, a VCN can be subdivided into one or more subnets. In one embodiment, a virtual router (VR) configured for the VCN (referred to as a VCN VR or simply a VR) enables communication between subnets within the VCN. For subnets within a VCN, the VR represents the logical gateway for that subnet, allowing the subnet (i.e., compute instances on that subnet) to communicate with endpoints on other subnets within the VCN and with other endpoints outside the VCN. A VCN VR is a logical entity configured to route traffic between VNICs within the VCN and the virtual gateway ("gateway") associated with the VCN. Gateways are described further below with respect to Figure 1. A VCN VR is a Layer 3 / IP layer concept. In one embodiment, there is one VCN VR for one VCN, and the VCN VR has an unlimited number of ports, which may be addressed by IP addresses, with one port for each subnet of the VCN. In this way, the VCN VR has a different IP address for each subnet within the VCN to which the VCN VR is connected. The VR is also connected to various gateways configured for the VCN. In one embodiment, a specific overlay IP address from a subnet's overlay IP address range is reserved for the VCN VR ports of that subnet. For example, consider a VCN that includes two subnets, each having the associated address ranges 10.0 / 16 and 10.1 / 16, respectively. For the first subnet in the VCN having the address range 10.0 / 16, addresses from this range are reserved for the VCN VR ports of that subnet. In some cases, the first IP address from this range may be reserved for the VCN VR. For example, for a subnet having the overlay IP address range 10.0 / 16, the IP address 10.0.0.1 may be reserved for the VCN VR ports of that subnet.For a second subnet within the same VCN having the address range 10.1 / 16, the VCN VR may have a port on that second subnet with the IP address 10.1.0.1. The VCN VR has a different IP address for each subnet within the VCN.

[0064] In some other embodiments, each subnet within a VCN may include a VR associated with itself, which is addressable by the subnet using a reserved or default IP address associated with the VR. The reserved or default IP address may be, for example, the first IP address from a range of IP addresses associated with that subnet. A VNIC within a subnet can communicate with the VR associated with the subnet using this default or reserved IP address (e.g., send and receive packets). In some such embodiments, the VR is the input / output point of that subnet. A VR associated with a subnet within a VCN can communicate with other VRs associated with other subnets within the VCN. A VR can also communicate with a gateway associated with the VCN. The VR functionality of a subnet operates on or by one or more NVDs that perform the VNIC functionality of the VNICs within the subnet.

[0065] Route tables, security rules, and DHCP options can be configured for a VCN. The route table is the VCN's virtual route table, containing rules for routing traffic from subnets within the VCN to destinations outside the VCN via a gateway or specially configured instance. The VCN's route table can be customized to control how packets are forwarded / routed to and from the VCN. DHCP options refer to configuration information automatically provided to an instance when it starts up.

[0066] Security rules configured for a VCN represent the VCN's overlay firewall rules. Security rules include ingress and egress rules and can specify the types of traffic (e.g., based on protocol and port) allowed to enter and exit instances within the VCN. Customers can choose whether specific rules are stateful or stateless. For example, a customer can allow incoming SSH traffic to a set of instances from any location by configuring a stateful ingress rule with source CIDR 0.0.0.0 / 0 and destination TCP port 22. Security rules can be implemented using network security groups or security lists. A network security group consists of a set of security rules that apply only to resources within that group. A security list, on the other hand, contains rules that apply to all resources within any subnet using the security list. A VCN may have a default security list containing default security rules. DHCP options configured for a VCN provide configuration information that is automatically provided to instances within the VCN when they start up.

[0067] In one embodiment, VCN configuration information is determined and stored by the VCN control plane. VCN configuration information may include, for example, information about address ranges associated with the VCN, subnets within the VCN and their associated information, one or more VRs associated with the VCN, compute instances within the VCN and their associated VNICs, NVDs (e.g., VNICs, VRs, gateways) that perform various virtualized network functions associated with the VCN, VCN status information, and other VCN-related information. In one embodiment, a VCN distribution service exposes the configuration information stored by the VCN control plane or a portion thereof to the NVD. The distributed information may be used to update information stored and used by the NVD to forward packets to and from compute instances within the VCN (e.g., forwarding tables, routing tables, etc.).

[0068] In one embodiment, the creation of VCNs and subnets is handled by a VCN control plane (CP), and the startup of compute instances is handled by the compute control plane. The compute control plane is responsible for allocating physical resources to compute instances and then calling the VCN control plane to create VNICs and connect them to the compute instances. The VCN CP also sends VCN data mappings to the VCN data plane, which is configured to perform packet forwarding and routing functions. In one embodiment, the VCN CP provides a distribution service responsible for providing updates to the VCN data plane. Examples of VCN control planes are also shown in Figures 16, 17, 18, and 19 (see references 1616, 1716, 1816, and 1916) and are described below.

[0069] A customer may create one or more VCNs using resources hosted by CSPI. Compute instances deployed in a customer's VCN may communicate with various endpoints. These endpoints may include those hosted by CSPI and those outside of CSPI.

[0070] Various different architectures for implementing cloud-based services using CSPI are shown in Figures 1, 2, 3, 4, 5, 16, 17, 18, and 20, and are described below. Figure 1 is a high-level diagram of a distributed environment 100 showing an overlay VCN or customer VCN hosted by CSPI according to one embodiment. The distributed environment shown in Figure 1 includes multiple components within the overlay network. The distributed environment 100 shown in Figure 1 is merely an example and is not intended to unduly limit the scope of the claimed embodiments. Many variations, alternatives, and modifications are possible. For example, in some implementations, the distributed environment shown in Figure 1 may include more or fewer systems or components than those shown in Figure 1, may combine two or more subsystems, or may include different configurations or arrangements of systems.

[0071] As shown in the example in Figure 1, the distributed environment 100 includes a CSPI 101 that provides services and resources that customers can subscribe to and use to build their own virtual cloud network (VCN). In one embodiment, the CSPI 101 provides IaaS services to the subscribing customer. The data centers within the CSPI 101 may be organized into one or more regions. One exemplary region, "Region US" 102, is shown in Figure 1. The customer constitutes the customer's VCN 104 in Region 102. The customer may deploy various compute instances to the VCN 104, which may include virtual machines or bare metal instances. Examples of instances include applications, databases, load balancers, etc.

[0072] In the embodiment shown in Figure 1, the customer's VCN 104 includes two subnets, namely "Subnet 1" and "Subnet 2," each subnet having its own CIDR IP address range. In Figure 1, the overlay IP address range for Subnet 1 is 10.0 / 16, and the address range for Subnet 2 is 10.1 / 16. The VCN Virtual Router (VCN VR) 105 represents the logical gateway of the VCN, enabling communication between subnets of VCN 104 and with other endpoints outside the VCN. The VCN VR 105 is configured to route traffic between VNICs within VCN 104 and gateways associated with VCN 104. The VCN VR 105 provides ports for each subnet of VCN 104. For example, the VCN VR 105 may provide a port with the IP address 10.0.0.1 for Subnet 1 and a port with the IP address 10.1.0.1 for Subnet 2.

[0073] Multiple compute instances may be deployed in each subnet, and compute instances may be virtual machine instances and / or bare metal instances. Compute instances within a subnet may be hosted by one or more host machines within CSPI101. Compute instances join the subnet via the VNIC associated with the compute instance. For example, as shown in Figure 1, compute instance C1 becomes part of subnet 1 via the VNIC associated with this compute instance. Similarly, compute instance C2 becomes part of subnet 1 via the VNIC associated with C2. In a similar manner, multiple compute instances, which may be virtual machine instances or bare metal instances, may become part of subnet 1. Each compute instance is assigned a private overlay IP address and MAC address via the VNIC associated with it. For example, in Figure 1, compute instance C1 has the overlay IP address 10.0.0.2 and the MAC address M1, while compute instance C2 has the private overlay IP address 10.0.0.3 and the MAC address M2. Each compute instance in subnet 1, including compute instances C1 and C2, has a default route to VCN VR105 using IP address 10.0.0.1, which is the IP address of the port of VCN VR105 in subnet 1.

[0074] Multiple compute instances, including virtual machine instances and / or bare metal instances, may be deployed in subnet 2. For example, as shown in Figure 1, compute instances D1 and D2 become part of subnet 2 via the VNICs associated with each compute instance. In the embodiment shown in Figure 1, compute instance D1 has the overlay IP address 10.1.0.2 and the MAC address of MM1, while compute instance D2 has the private overlay IP address 10.1.0.3 and the MAC address of MM2. Each compute instance in subnet 2, including compute instances D1 and D2, has a default route to VCN VR105 using the IP address 10.1.0.1, which is the IP address of the port of VCN VR105 in subnet 2.

[0075] VCN A104 may also include one or more load balancers. For example, a load balancer may be provided to a subnet and configured to load balance traffic across multiple compute instances on the subnet. A load balancer may also be provided to load balance traffic across multiple subnets within the VCN.

[0076] A specific compute instance deployed in VCN104 can communicate with various endpoints. These endpoints may include endpoints hosted by CSPI200 and endpoints outside of CSPI200. Endpoints hosted by CSPI101 may include endpoints on the same subnet as a particular compute instance (e.g., communication between two compute instances in subnet 1), endpoints on different subnets but within the same VCN (e.g., communication between a compute instance in subnet 1 and a compute instance in subnet 2), endpoints in different VCNs within the same region (e.g., communication between a compute instance in subnet 1 and an endpoint in the same VCN in region 106 or 110, or between a compute instance in subnet 1 and an endpoint in service network 110 within the same region), or endpoints in VCNs in different regions (e.g., communication between a compute instance in subnet 1 and an endpoint in the same VCN in region 108). A compute instance in a subnet hosted by CSPI101 can also communicate with endpoints not hosted by CSPI101 (i.e., outside of CSPI101). These external endpoints include endpoints within the customer's on-premises network 116, endpoints within networks 118 hosted by other remote clouds, public endpoints 114 accessible via public networks such as the Internet, and other endpoints.

[0077] Communication between compute instances on the same subnet is facilitated using VNICs associated with the source and destination compute instances. For example, compute instance C1 in subnet 1 may want to send a packet to compute instance C2, also in subnet 1. For a packet originating from the source compute instance and destined for another compute instance in the same subnet, the packet is first processed by the VNIC associated with the source compute instance. The processing performed by the VNIC associated with the source compute instance may include determining the packet's destination information from the packet header, identifying any policies (e.g., security lists) configured for the VNIC associated with the source compute instance, determining the packet's next hop, performing any packet encapsulation / decapsulation functions as needed, and then forwarding / routing the packet to the next hop to facilitate communication of the packet with its intended destination. If the destination compute instance is in the same subnet as the source compute instance, the VNIC associated with the source compute instance is configured to identify the VNIC associated with the destination compute instance and forward the packet to that VNIC for processing. The VNIC associated with the destination compute instance then executes and forwards the packet to the destination compute instance.

[0078] For packets propagating from a compute instance within a subnet to an endpoint in a different subnet within the same VCN, this communication is facilitated by the VNICs and VCN VRs associated with the source and destination compute instances. For example, if compute instance C1 in subnet 1 in Figure 1 wants to send a packet to compute instance D1 in subnet 2, this packet is first processed by the VNIC associated with compute instance C1. The VNIC associated with compute instance C1 is configured to route the packet to VCN VR105 using the VCN VR's default route or port 10.0.0.1. VCN VR105 is configured to route the packet to subnet 2 using port 10.1.0.1. Next, the packet is received and processed by the VNIC associated with D1, and the VNIC forwards the packet to compute instance D1.

[0079] For packets propagated from compute instances within VCN104 to endpoints outside VCN104, communication is facilitated by the VNIC associated with the source compute instance, VCN VR105, and gateways associated with VCN104. One or more types of gateways may be associated with VCN104. A gateway is an interface between a VCN and another endpoint, which is outside the VCN. A gateway is a Layer 3 / IP layer concept that enables a VCN to communicate with endpoints outside the VCN. Thus, gateways facilitate traffic flow between a VCN and other VCNs or networks. Different types of gateways can be configured for a VCN to facilitate different types of communication with different types of endpoints. Depending on the gateway, communication may traverse a public network (e.g., the Internet) or a private network. Different communication protocols may be used for these communications.

[0080] For example, compute instance C1 may want to communicate with an endpoint outside of VCN104. The packet may first be processed by the VNIC associated with source compute instance C1. This VNIC processing determines that the packet's destination is outside of C1's subnet 1. The VNIC associated with C1 may forward the packet to VCN VR105 of VCN104. VCN VR105 then processes the packet and, as part of this processing, determines a specific gateway associated with VCN104 as the packet's next hop based on the packet's destination. VCN VR105 may then forward the packet to a specific identified gateway. For example, if the destination is an endpoint within a customer's on-premises network, VCN VR105 may forward the packet to a Dynamic Routing Gateway (DRG) gateway 122 configured for VCN104. The packet may then be forwarded from the gateway to the next hop to facilitate its delivery to its final intended destination.

[0081] Various types of gateways can be configured for a VCN. An example of a gateway that can be configured for a VCN is shown in Figure 1 and described below. Examples of gateways associated with a VCN are also shown in Figures 16, 17, 18, and 19 (for example, gateways referenced by reference numbers 1634, 1636, 1638, 1734, 1736, 1738, 1834, 1836, 1838, 1934, 1936, and 1938) and described below. As shown in the embodiment shown in Figure 1, a dynamic routing gateway (DRG) 122 may be added to or associated with a customer's VCN 104 to provide a route for private network traffic communication between the customer's VCN 104 and another endpoint, where this other endpoint may be the customer's on-premises network 116, a VCN 108 in a different region of CSPI 101, or another remote cloud network 118 not hosted by CSPI 101. A customer's on-premises network 116 may be the customer's network or data center built using the customer's resources. Access to the customer's on-premises network 116 is typically extremely restricted. For a customer who has both the customer's on-premises network 116 and one or more VCNs 104 deployed or hosted in the cloud by CSPI 101, the customer may want the customer's on-premises network 116 and the customer's cloud-based VCNs 104 to be able to communicate with each other. This would allow the customer to build an extended hybrid environment encompassing the customer's VCNs 104 hosted by CSPI 101 and the customer's on-premises network 116. DRG 122 enables this communication. To enable such communication, a communication channel 124 is configured, with one endpoint of this channel located within the customer's on-premises network 116 and the other endpoints located within CSPI 101 and connected to the customer's VCNs 104. The communication channel 124 can be via a public communication network such as the internet or a private communication network.Various communication protocols can be used, such as IPsec VPN technology which uses public communication networks like the Internet, and Oracle's FastConnect technology which uses a private network instead of a public network. A device or equipment within the customer's on-premises network 116 that forms one endpoint of communication channel 124 is called customer premise equipment (CPE), such as CPE126 shown in Figure 1. On the CSPI101 side, the endpoint may be a host machine running DRG122.

[0082] In one embodiment, a Remote Peering Connection (RPC) may be added to the DRG, enabling a customer to peer one VCN with another VCN in a different region. Using such an RPC, a customer's VCN 104 can connect to a VCN 108 in a different region using the DRG 122. The DRG 122 may also be used to communicate with other remote cloud networks 118 not hosted by the CSPI 101, such as Microsoft® Azure cloud or Amazon AWS cloud.

[0083] As shown in Figure 1, an Internet Gateway (IGW) 120 may be configured for a customer's VCN104, enabling compute instances on VCN104 to communicate with a public endpoint 114 accessible via a public network such as the Internet. The IGW 120 is a gateway that connects the VCN to a public network such as the Internet. The IGW 120 enables direct access from public subnets within a VCN, such as VCN104 (resources within public subnets have public overlay IP addresses), to public endpoints 112 on the public network 114, such as the Internet. Connections can be initiated using the IGW 120 from subnets within VCN104 or from the Internet.

[0084] A Network Address Translation (NAT) gateway 128 is configured for the customer's VCN 104 to enable internet access for cloud resources within the customer's VCN that do not have dedicated public overlay IP addresses. The NAT gateway 128 enables such access without directly exposing those resources to incoming internet connections (e.g., L4-L7 connections). This allows private access from private subnets within the VCN, such as private subnet 1 within VCN 104, to public endpoints on the internet. The NAT gateway can only initiate connections from private subnets to the public internet; it cannot initiate connections from the internet to private subnets.

[0085] In one embodiment, a Service Gateway (SGW) 126 may be configured for a customer's VCN 104, providing a route for private network traffic between the VCN 104 and supported service endpoints within a service network 110. In one embodiment, the service network 110 may be provided by a Customer Service Provider (CSP) and may provide a variety of services. An example of such a service network is Oracle's Services Network, which provides a variety of services that can be used by customers. For example, compute instances (e.g., database systems) within a private subnet of a customer's VCN 104 can back up data to a service endpoint (e.g., object storage) without requiring a public IP address or internet access. In one embodiment, a VCN may contain only one SGW, and connections may be initiated only from subnets within the VCN and not from the service network 110. If a VCN is peered with another VCN, resources in the other VCN typically cannot access the SGW. Resources in an on-premises network connected to a VCN using FastConnect or VPN connections may also use the service gateway configured for that VCN.

[0086] In one implementation, SGW126 uses the concept of service-classless inter-domain routing (CIDR) labels, which are strings representing the public IP address range for all regions of the target service or group of services. Customers use service CIDR labels when configuring SGW and associated route rules to control traffic to their services. Optionally, customers can also utilize service CIDR labels when configuring security rules, eliminating the need to adjust those security rules if the public IP addresses of their services change in the future.

[0087] A Local Peering Gateway (LPG) 132 is a gateway that can be added to a customer's VCN 104, enabling VCN 104 to peer with another VCN within the same region. Peering means that VCNs communicate using private IP addresses without traffic traversing public networks such as the internet or routing traffic through the customer's on-premises network 116. In a preferred embodiment, a VCN includes a separate LPG for each peering it establishes. Local peering, or VCN peering, is a common method used to establish network connectivity between different applications or infrastructure management functions.

[0088] Service providers, such as service providers within service network 110, can provide access to their services using various access models. According to the public access model, a service may be exposed as a public endpoint, which can be publicly accessible by compute instances within the customer's VCN via a public network such as the internet, and / or privately accessible via SGW126. According to a specific private access model, a service may be made accessible as a private IP endpoint within a private subnet within the customer's VCN. This access is called private endpoint (PE) access and allows service providers to expose their services as instances within the customer's private network. A private endpoint resource represents a service within the customer's VCN. Each PE appears as a VNIC (called a PE-VNIC, having one or more private IPs) within a subnet selected by the customer within the customer's VCN. Thus, a PE provides a way to present services within a private subnet of the customer's VCN using VNICs. Because the endpoint is exposed as a VNIC, all features associated with the VNIC, such as routing rules and security lists, are available to the PE VNIC.

[0089] A service provider can register a service to enable access via a Public Access Point (PE). The provider can associate policies with the service that restrict its visibility to customer tenancies. A provider can register multiple services under a single virtual IP address (VIP), especially in the case of multi-tenant services. Multiple such private endpoints (within multiple VCNs) may exist representing the same service.

[0090] Next, compute instances within a private subnet can access the service using the private IP address of the PE VNIC or the DNS name of the service. Compute instances within a customer's VCN can access the service by sending traffic to the private IP address of the PE within the customer's VCN. The Private Access Gateway (PAGW) 130 is a gateway resource that can be connected to the service provider's VCN (e.g., a VCN within service network 110) and acts as the input / output point for all traffic to and from the private endpoint in the customer's subnet. The PAGW 130 allows the provider to scale the number of PE connections without utilizing internal IP address resources. The provider only needs to configure one PAGW for any number of services registered within a single VCN. The provider can represent a service as a private endpoint in multiple VCNs for one or more customers. From the customer's perspective, the PE VNIC appears to be connected to the service the customer wishes to interact with, rather than to the customer's instance. Traffic destined for the private endpoint is routed to the service via the PAGW 130. These are called customer-to-service (C2S) connections.

[0091] The PE concept can also be used to extend private access to a service to the customer's on-premises network and data center by allowing traffic to flow through FastConnect / IPsec links and private endpoints within the customer's VCN. Private access to the service can also be extended to the customer's peered VCN by allowing traffic to flow between LPG132 and the PE within the customer's VCN.

[0092] Customers can control routing within their VCN at the subnet level, allowing them to specify which subnets within their VCN, such as VCN104, use which gateways. The VCN's route table is used to determine whether traffic is allowed to exit the VCN through a particular gateway. For example, in a specific case, the route table for a public subnet within a customer's VCN104 might send non-local traffic through IGW120. The route table for a private subnet within the same customer's VCN104 might send traffic destined for CSP services through SGW126. All remaining traffic could be sent through NAT gateway 128. The route table only controls traffic leaving the VCN.

[0093] Security lists associated with a VCN are used to control traffic entering the VCN via a gateway through inbound connections. All resources within a subnet use the same route table and security list. Security lists can be used to control specific types of traffic that are allowed to enter and leave instances within a VCN subnet. Security list rules can include inbound and outbound rules. For example, inbound rules may specify allowed source address ranges, while outbound rules may specify allowed destination address ranges. Security rules may specify specific protocols (e.g., TCP, ICMP), specific ports (e.g., 22 for SSH, 3389 for Windows® RDP), etc. In some implementations, the instance's operating system may enforce its own firewall rules that match the security list rules. Rules can be stateful (e.g., connections are tracked and responses are automatically allowed without using explicit security list rules for response traffic) or stateless.

[0094] Access from a customer's VCN (i.e., by resources or compute instances deployed on VCN104) can be classified as public access, private access, or dedicated access. Public access refers to an access model in which a public IP address or NAT is used to access a public endpoint. Private access allows a customer's workload within VCN104 with a private IP address (e.g., resources in a private subnet) to access a service without traversing a public network such as the internet. In one embodiment, CSPI101 allows a customer's VCN workload with a private IP address to access a service (or its public service endpoint) using a service gateway. Thus, the service gateway provides a private access model by establishing a virtual link between the customer's VCN and the public endpoint of the service that resides outside the customer's private network.

[0095] Furthermore, CSPI may provide dedicated public access technology using technologies such as FastConnect public peering, which allows a customer's on-premises instances to access one or more services within the customer's VCN using FastConnect connectivity without traversing public networks such as the internet. CSPI may also provide dedicated private access using FastConnect private peering, which allows a customer's on-premises instances with private IP addresses to access workloads in the customer's VCN using FastConnect connectivity. FastConnect is a network connectivity alternative to using the public internet for connecting a customer's on-premises network to CSPI and its services. FastConnect provides an easy, resilient, and economical way to create dedicated private connectivity with higher bandwidth options and a more reliable and consistent network experience compared to internet-based connectivity.

[0096] Figure 1 and the accompanying explanation above illustrate the various virtual components within an exemplary virtual network. As previously mentioned, the virtual network is built on top of an underlying physical network or infrastructure network. Figure 2 shows a simplified architectural diagram of the physical components in the physical network within the underlying CSPI200 for the virtual network, according to one embodiment. As shown in the figure, the CSPI200 provides a distributed environment that includes components and resources (e.g., compute resources, memory resources, and network resources) provided by a Cloud Service Provider (CSP). These components and resources are used to deliver cloud services (e.g., IaaS services) to subscribers, i.e., customers who subscribe to one or more services provided by the CSP. Based on the services subscribed by the customer, a subset of the CSPI200's resources (e.g., compute resources, memory resources, and network resources) is provisioned for the customer. The customer can then use the physical compute resources, memory resources, and network resources provided by the CSPI200 to build their own cloud-based (i.e., CSPI-hosted) customizable private virtual network. As already indicated, these customer networks are called virtual cloud networks (VCNs). Customers can deploy one or more customer resources, such as compute instances, to these customer VCNs. Compute instances can take the form of virtual machines, bare metal instances, etc. CSPI200 provides infrastructure and a suite of complementary cloud services that enable customers to build and run a wide range of applications and services in a highly available hosted environment.

[0097] In the embodiment shown in Figure 2, the physical components of the CSPI200 include one or more physical host machines or physical servers (e.g., 202, 206, 208), network virtualization devices (NVDs) (e.g., 210, 212), top-of-rack (TOR) switches (e.g., 214, 216), and a physical network (e.g., 218), and switches within the physical network 218. The physical host machines or physical servers may host and run various compute instances participating in one or more subnets of the VCN. Compute instances may include virtual machine instances and bare metal instances. For example, the various compute instances shown in Figure 1 may be hosted by the physical host machines shown in Figure 2. Virtual machine compute instances in the VCN may run by one host machine or by several different host machines. The physical host machines may also host virtual host machines, container-based hosts or functions, etc. The VNIC and VCN VR shown in Figure 1 may run by the NVDs shown in Figure 2. The gateway shown in Figure 1 can be run by the host machine and / or by the NVD shown in Figure 2.

[0098] A host machine or server may run a hypervisor (also known as a virtual machine monitor or VMM) that creates and activates virtual environments on the host machine. Virtualization environments facilitate cloud-based computing. On a host machine, one or more compute instances may be created, run, and managed by the hypervisor on that host machine. The hypervisor on the host machine enables the sharing of the host machine's physical computing resources (e.g., compute resources, memory resources, and network resources) among the various compute instances running on the host machine.

[0099] For example, as shown in Figure 2, host machines 202 and 208 run hypervisors 260 and 266, respectively. These hypervisors can be implemented using software, firmware, hardware, or a combination thereof. Typically, a hypervisor is a process or software layer that resides on top of the host machine's operating system (OS), which runs on the host machine's hardware processors. A hypervisor provides a virtual environment by enabling the sharing of the host machine's physical computing resources (e.g., processing resources such as processors / cores, memory resources, and network resources) among various virtual machine computing instances run by the host machine. For example, in Figure 2, hypervisor 260 may reside on top of the host machine 202's OS, enabling the sharing of the host machine 202's computing resources (e.g., processing resources, memory resources, and network resources) among computing instances (e.g., virtual machines) run by the host machine 202. A virtual machine may have its own operating system (called a guest operating system), which may be the same as or different from the host machine's OS. The operating system of a virtual machine running on a host machine may be the same as, or different from, the operating system of another virtual machine running on the same host machine. Therefore, a hypervisor allows multiple operating systems to run in parallel with each other while sharing the same computing resources on the host machine. The host machines shown in Figure 2 may have the same or different types of hypervisors.

[0100] Compute instances can be virtual machine instances or bare metal instances. In Figure 2, compute instance 268 on host machine 202 and compute instance 274 on host machine 208 are examples of virtual machine instances. Host machine 206 is an example of a bare metal instance provided to a customer.

[0101] In one example, an entire host machine may be provisioned to a single customer, and all one or more compute instances (either virtual machines or bare metal instances) hosted by that host machine belong to that same customer. In another example, a host machine may be shared among multiple customers (i.e., multiple tenants). In such a multi-tenancy scenario, the host machine may host virtual machine compute instances belonging to different customers. These compute instances may be members of different VCNs of different customers. In one embodiment, bare metal compute instances are hosted by bare metal servers that do not have a hypervisor. When bare metal compute instances are provisioned, a single customer or tenant maintains control of the physical CPU, memory, and network interfaces of the host machine hosting the bare metal instances, and the host machine is not shared with other customers or tenants.

[0102] As mentioned above, each compute instance that is part of a VCN is associated with a VNIC that enables the compute instance to be a member of the VCN's subnet. The VNIC associated with a compute instance facilitates the communication of packets or frames to and from the compute instance. The VNIC is associated with a compute instance when the compute instance is created. In one embodiment, for a compute instance run by a host machine, the VNIC associated with that compute instance is run by an NVD connected to the host machine. For example, in Figure 2, host machine 202 runs a virtual machine compute instance 268 associated with VNIC 276, and VNIC 276 is run by an NVD 210 connected to host machine 202. In another example, a bare metal instance 272 hosted by host machine 206 is associated with VNIC 280, which is run by an NVD 212 connected to host machine 206. In yet another example, VNIC 284 is associated with compute instance 274 run by host machine 208, and VNIC 284 is run by an NVD 212 connected to host machine 208.

[0103] For compute instances hosted by a host machine, the NVDs connected to that host machine also run the VCN VRs corresponding to the VCNs to which those compute instances are members. For example, in the embodiment shown in Figure 2, NVD210 runs VCN VR277 corresponding to the VCN to which compute instance 268 is a member. NVD212 may also run one or more VCN VRs283 corresponding to the VCNs to which compute instances hosted by host machines 206 and 208 are associated.

[0104] A host machine may include one or more network interface cards (NICs) that enable the host machine to connect to other devices. The NICs on the host machine may provide one or more ports (or interfaces) that enable the host machine to connect communicatively to another device. For example, a host machine may connect to an NVD using one or more ports (or interfaces) provided on the host machine and the NVD. A host machine may also connect to other devices, such as another host machine.

[0105] For example, in Figure 2, host machine 202 is connected to NVD210 using a link 220 extending between port 234 provided by NIC 232 of host machine 202 and port 236 of NVD210. Host machine 206 is connected to NVD212 using a link 224 extending between port 246 provided by NIC 244 of host machine 206 and port 248 of NVD212. Host machine 208 is connected to NVD212 using a link 226 extending between port 252 provided by NIC 250 of host machine 208 and port 254 of NVD212.

[0106] Next, the NVDs are connected via communication links to top-of-rack (TOR) switches (also called switch fabrics) connected to the physical network 218. In some embodiments, the links between the host machines and the NVDs, and between the NVDs and the TOR switches, are Ethernet® links. For example, in Figure 2, links 228 and 230 are used to connect NVDs 210 and 212 to TOR switches 214 and 216, respectively. In some embodiments, links 220, 224, 226, 228, and 230 are Ethernet® links. The collection of host machines and NVDs connected to the TOR is sometimes referred to as a rack.

[0107] The physical network 218 provides a communication fabric that allows TOR switches to communicate with each other. The physical network 218 can be a multi-layer network. In one implementation, the physical network 218 is a multi-layer Clos network of switches, and TOR switches 214 and 216 represent leaf-level nodes in a multi-layer multi-node physical switching network 218. Various Clos network configurations are possible, including but not limited to 2-layer, 3-layer, 4-layer, 5-layer networks, and generally "n"-layer networks. An example of a Clos network is shown in Figure 5 and described below.

[0108] Various connection configurations are possible between the host machine and the NVD, including one-to-one, many-to-one, and one-to-many configurations. In a one-to-one configuration, each host machine is connected to its own separate NVD. For example, in Figure 2, host machine 202 is connected to NVD210 via host machine 202's NIC232. In a many-to-one configuration, multiple host machines are connected to a single NVD. For example, in Figure 2, host machines 206 and 208 are connected to the same NVD212, respectively, via NIC244 and 250.

[0109] In a one-to-many configuration, one host machine is connected to multiple NVDs. Figure 3 shows an example within CSPI300 where a host machine is connected to multiple NVDs. As shown in Figure 3, the host machine 302 has a network interface card (NIC) 304 with multiple ports 306 and 308. The host machine 300 is connected to a first NVD 310 via port 306 and link 320, and to a second NVD 312 via port 308 and link 322. Ports 306 and 308 may be Ethernet® ports, and links 320 and 322 between the host machine 302 and the NVDs 310 and 312 may be Ethernet® links. Next, the NVD 310 is connected to a first TOR switch 314, and the NVD 312 is connected to a second TOR switch 316. The links between the NVDs 310 and 312 and the TOR switches 314 and 316 may be Ethernet® links. TOR switches 314 and 316 represent Layer 0 switching devices in a multilayer physical network 318.

[0110] The configuration shown in Figure 3 provides two separate physical network paths to the host machine 302 across the physical switch network 318: a first path that crosses TOR switch 314 and passes through NVD 310 to the host machine 302, and a second path that crosses TOR switch 316 and passes through NVD 312 to the host machine 302. These separate paths result in improved availability (referred to as high availability) of the host machine 302. If there is a problem with one of the paths (e.g., one link in the path fails) or a device (e.g., a particular NVD is not functioning), the other path can be used for communication with the host machine 302.

[0111] In the configuration shown in Figure 3, the host machine connects to two different NVDs using two different ports provided by the host machine's NIC. In other embodiments, the host machine may include multiple NICs that enable the host machine to connect to multiple NVDs.

[0112] Referring again to Figure 2, an NVD is a physical device or component that performs one or more network and / or storage virtualization functions. An NVD can be any device having one or more processing units (e.g., a CPU, Network Processing Units (NPUs), FPGA, packet processing pipeline, etc.), memory including a cache, and ports. Various virtualization functions can be performed by the software / firmware executed by one or more processing units of the NVD.

[0113] NVDs can be implemented in various forms. For example, in one embodiment, an NVD is implemented as an interface card called a smart NIC or intelligent NIC that incorporates an embedded processor. A smart NIC is a separate device from the NIC on the host machine. In Figure 2, NVDs 210 and 212 may be implemented as smart NICs connected to host machine 202, and to host machines 206 and 208, respectively.

[0114] However, a smart NIC is merely one example of an NVD implementation. Various other implementations are possible. For example, in some other implementations, the NVD or one or more functions performed by the NVD may be incorporated into or performed by one or more host machines, one or more TOR switches, and other components of the CSPI200. For instance, the NVD may be embodied in a host machine, and the functions performed by the NVD are performed by the host machine. Another example is that the NVD may be part of a TOR switch, or the TOR switch may be configured to perform the functions performed by the NVD, thereby enabling the TOR switch to perform various complex packet translations used in public clouds. A TOR that performs the functions of the NVD is sometimes called a smart TOR. In yet another implementation where virtual machine (VM) instances rather than bare metal (BM) instances are provided to customers, the functions performed by the NVD may be implemented within the host machine's hypervisor. In some other implementations, some of the NVD's functions may be offloaded to a centralized service running on a set of host machines.

[0115] In some embodiments, such as when implemented as a smart NIC as shown in Figure 2, the NVD may have multiple physical ports that enable the NVD to connect to one or more host machines and one or more TOR switches. The ports on the NVD may be classified as host-facing ports (also called “south ports”) or network-facing or TOR-facing ports (also called “north ports”). Host-facing ports on the NVD are the ports used to connect the NVD to a host machine. An example of host-facing ports in Figure 2 includes port 236 on the NVD210, and ports 248 and 254 on the NVD212. Network-facing ports on the NVD are the ports used to connect the NVD to a TOR switch. An example of network-facing ports in Figure 2 includes port 256 on the NVD210 and port 258 on the NVD212. As shown in Figure 2, the NVD210 connects to the TOR switch 214 using a link 228 extending from port 256 on the NVD210 to the TOR switch 214. Similarly, the NVD212 is connected to the TOR switch 216 using a link 230 that extends from port 258 of the NVD212 to the TOR switch 216.

[0116] The NVD may receive packets and frames from a host machine via its host-facing port (e.g., packets and frames generated by compute instances hosted by the host machine), perform the necessary packet processing, and then forward those packets and frames to the TOR switch via the NVD's network-facing port. The NVD may also receive packets and frames from the TOR switch via its network-facing port, perform the necessary packet processing, and then forward those packets and frames to the host machine via the NVD's host-facing port.

[0117] In one embodiment, there may be multiple ports and associated links between the NVD and the TOR switch. These ports and links can be aggregated to form a link aggregator group (LAG) of multiple ports or links. Link aggregation allows multiple physical links between two endpoints (e.g., between the NVD and the TOR switch) to be treated as a single logical link. All physical links within a given LAG can operate in full-duplex mode at the same speed. LAGs help increase bandwidth and improve the reliability of the connection between the two endpoints. If one of the physical links in the LAG fails, traffic is dynamically and transparently reassigned to one of the other physical links in the LAG. Aggregated physical links provide higher bandwidth than individual links. Multiple ports associated with a LAG are treated as a single logical port. Traffic can be load-balanced across the multiple physical links of the LAG. One or more LAGs can be configured between two endpoints. The two endpoints may be between the NVD and the TOR switch, between a host machine and the NVD, etc.

[0118] NVD implements or performs network virtualization functions. These functions are performed by software / firmware run by NVD. Examples of network virtualization functions include, but are not limited to, packet encapsulation and decapsulation functions, functions for creating VCN networks, functions for enforcing network policies such as VCN security list (firewall) functions, and functions for facilitating the routing and forwarding of packets to and from compute instances within the VCN. In one embodiment, upon receiving a packet, NVD is configured to run a packet processing pipeline to process the packet and determine how the packet should be forwarded or routed. As part of this packet processing pipeline, NVD may run one or more virtual functions associated with an overlay network, such as running a VNIC associated with compute instances within the VCN, running a virtual router (VR) associated with the VCN, encapsulating and decapsulating packets to facilitate forwarding or routing within the virtual network, running a gateway (e.g., a local peering gateway), enforcing security lists and network security groups, network address translation (NAT) functions (e.g., translation from public IP to private IP on a per-host basis), throttling functions, and other functions.

[0119] In one embodiment, the packet processing data path within the NVD may comprise multiple packet pipelines, each consisting of a series of packet translation stages. In one implementation, upon receiving a packet, it is parsed and classified into a single pipeline. The packet is then processed linearly, one stage at a time, until it is either discarded or transmitted through the NVD's interface. These stages provide basic functional packet processing components (e.g., header validation, bandwidth adjustment, insertion of new Layer 2 headers, L4 firewall implementation, VCN encapsulation / decapsulation, etc.) so that new pipelines can be constructed by assembling existing stages, and new functionality can be added by creating new stages and inserting them into existing pipelines.

[0120] The NVD can perform both control plane and data plane functions corresponding to the VCN's control plane and data plane. Examples of the VCN control plane are also shown in Figures 16, 17, 18, and 19 (see references 1616, 1716, 1816, and 1916) and are described below. Examples of the VCN data plane are also shown in Figures 16, 17, 18, and 19 (see references 1618, 1718, 1818, and 1918) and are described below. Control plane functions include functions used to configure the network that controls how data is forwarded (e.g., setting routes and route tables, configuring VNICs, etc.). In one embodiment, a VCN control plane is provided that centrally calculates the mapping between all overlays and the substrate and exposes those mappings to the NVD and to various virtual network edge devices such as gateways such as DRG, SGW, IGW, etc. Firewall rules may be exposed using the same mechanism. In one embodiment, the NVD retrieves only the mappings relevant to that NVD. The data plane functionality includes the ability to actually route / forward packets based on configuration settings that use the control plane. The VCN data plane is implemented by encapsulating customer network packets before they traverse the underlying network. The encapsulation / decapsulation function is implemented by the NVD. In one embodiment, the NVD is configured to intercept all network packets entering and leaving the host machine and to perform network virtualization functions.

[0121] As shown above, NVDs perform various virtualization functions, including VNICs and VCN VRs. An NVD can run a VNIC associated with a compute instance hosted by one or more host machines connected to that VNIC. For example, as shown in Figure 2, NVD210 runs the functions of VNIC276 associated with compute instance 268 hosted by host machine 202 connected to NVD210. As another example, NVD212 runs VNIC280 associated with bare-metal compute instance 272 hosted by host machine 206 and VNIC284 associated with compute instance 274 hosted by host machine 208. A host machine can host compute instances belonging to different VCNs belonging to different customers, and an NVD connected to a host machine can run the VNICs corresponding to those compute instances (i.e., run the functions associated with the VNICs).

[0122] An NVD also runs a VCN virtual router corresponding to the VCN of a compute instance. For example, in the embodiment shown in Figure 2, NVD210 runs VCN VR277 corresponding to the VCN to which compute instance 268 belongs. NVD212 runs one or more VCN VR283 corresponding to one or more VCNs to which compute instances hosted by host machines 206 and 208 belong. In one embodiment, the VCN VR corresponding to that VCN is run by all NVDs connected to the host machine hosting at least one compute instance belonging to that VCN. If a host machine hosts compute instances belonging to different VCNs, the NVDs connected to that host machine may run VCN VRs corresponding to those different VCNs.

[0123] In addition to VNICs and VCN VRs, an NVD may include one or more hardware components that can run various software (e.g., daemons) and facilitate various network virtualization functions performed by the NVD. For simplicity, these various components are grouped together as a “packet processing component” as shown in Figure 2. For example, NVD210 has a packet processing component 286, and NVD212 has a packet processing component 288. For example, the NVD’s packet processing component may include a packet processor configured to interact with the NVD’s ports and hardware interfaces to monitor all packets received by the NVD and packets transmitted using the NVD, and to store network information. Network information may include, for example, network flow information that identifies various network flows processed by the NVD and per-flow information (e.g., per-flow statistics). In one embodiment, network flow information may be stored per VNIC. In addition to performing per-packet operations, the packet processor may implement stateful NAT and L4 firewall (FW). As another example, the packet processing component may include a replication agent configured to replicate information stored by the NVD to one or more different replication target stores. Yet another example, the packet processing component may include a logging agent configured to perform the NVD's logging function. The packet processing component may also include software for monitoring the performance and health of the NVD, and possibly software for monitoring the status and health of other components connected to the NVD.

[0124] Figure 1 shows components of an exemplary virtual network or overlay network, including a VCN, subnets within the VCN, compute instances deployed on the subnets, VNICs associated with the compute instances, a VCN VR, and a set of gateways configured for the VCN. The overlay components shown in Figure 1 may run or host one or more of the physical components shown in Figure 2. For example, compute instances within a VCN may run or host one or more host machines shown in Figure 2. In the case of compute instances hosted by a host machine, the VNIC associated with that compute instance is typically run by an NVD connected to that host machine (i.e., the VNIC functionality is provided by the NVD connected to that host machine). The VCN VR functionality of the VCN is run by all NVDs connected to the host machines that host or run compute instances that are part of that VCN. Gateways associated with a VCN may run by one or more different types of NVDs. For example, one gateway may run by a smart NIC, while others may run by one or more host machines or other implementations of NVDs.

[0125] As mentioned earlier, compute instances within a customer's VCN can communicate with various endpoints, which may be on the same subnet as the source compute instance, on a different subnet within the same VCN as the source compute instance, or outside the source compute instance's VCN. These communications are facilitated using the VNIC associated with the compute instance, the VCN VR, and the gateway associated with the VCN.

[0126] For communication between two compute instances on the same subnet within a VCN, communication is facilitated using VNICs associated with the source and destination compute instances. The source and destination compute instances may be hosted by the same host machine or by different host machines. Packets originating from the source compute instance may be forwarded from the host machine hosting the source compute instance to an NVD connected to that host machine. In the NVD, packets are processed using a packet processing pipeline, which may include running the VNIC associated with the source compute instance. Because the destination endpoint of the packet is within the same subnet, running the VNIC associated with the source compute instance causes the packet to be forwarded to an NVD running the VNIC associated with the destination compute instance, which then processes the packet and forwards it to the destination compute instance. The VNICs associated with source and destination compute instances can run on the same NVD (for example, if both source and destination compute instances are hosted by the same host machine) or on different NVDs (for example, if source and destination compute instances are hosted by different host machines connected to different NVDs). The VNICs can use routing / forwarding tables stored by the NVD to determine the next hop of a packet.

[0127] For packets propagated from a compute instance within a subnet to an endpoint in a different subnet within the same VCN, the packet originating from the source compute instance is propagated from the host machine hosting the source compute instance to the NVD connected to that host machine. The NVD processes the packet using a packet processing pipeline, which can include one or more VNICs and the execution of a VR associated with the VCN. For example, as part of the packet processing pipeline, the NVD executes or invokes a function corresponding to the VNIC associated with the source compute instance (also called executing the VNIC). The function executed by the VNIC may include examining the VLAN tag on the packet. Because the packet's destination is outside the subnet, the VCN VR function is then invoked and executed by the NVD. The VCN VR then routes the packet to the NVD running the VNIC associated with the destination compute instance. The VNIC associated with the destination compute instance then processes the packet and forwards it to the destination compute instance. The VNICs associated with the source and destination compute instances can run on the same NVD (for example, if both the source and destination compute instances are hosted by the same host machine) or on different NVDs (for example, if the source and destination compute instances are hosted by different host machines connected to different NVDs).

[0128] If the destination of a packet is outside the VCN of the source compute instance, the packet originating from the source compute instance is propagated from the host machine hosting the source compute instance to the NVD connected to that host machine. The NVD runs the VNIC associated with the source compute instance. Because the destination endpoint of the packet is outside the VCN, the packet is then processed by the VCN VR of that VCN. The NVD may invoke the VCN VR function, causing the packet to be forwarded to an NVD running the appropriate gateway associated with the VCN. For example, if the destination is an endpoint within the customer's on-premises network, the packet may be forwarded by the VCN VR to an NVD running the DRG gateway configured for the VCN. The VCN VR can run on the same NVD running the VNIC associated with the source compute instance, or by a different NVD. The gateway can run by an NVD, which could be a smart NIC, a host machine, or another NVD implementation. The packet is then processed by the gateway and forwarded to the next hop, which facilitates the propagation of the packet to the intended destination endpoint. For example, in the embodiment shown in Figure 2, packets originating from compute instance 268 may be propagated from host machine 202 to NVD210 via link 220 (using NIC 232). On NVD210, VNIC 276 is invoked because it is the VNIC associated with source compute instance 268. VNIC 276 is configured to examine the encapsulated information in the packet, determine the next hop for forwarding the packet to facilitate its delivery to the intended destination endpoint, and then forward the packet to the determined next hop.

[0129] Compute instances deployed on a VCN can communicate with various endpoints. These endpoints may include endpoints hosted by CSPI200 and endpoints outside of CSPI200. Endpoints hosted by CSPI200 may include instances within the same VCN or other VCNs, which may be customer VCNs or VCNs not belonging to a customer. Communication between endpoints hosted by CSPI200 may be performed over the physical network 218. Compute instances may also communicate with endpoints that are not hosted by CSPI200 or are outside of CSPI200. Examples of these endpoints include endpoints within the customer's on-premises network or data center, or public endpoints accessible over a public network such as the Internet. Communication with endpoints outside of CSPI200 may be performed over a public network (e.g., the Internet) (not shown in Figure 2) or a private network (not shown in Figure 2) using various communication protocols.

[0130] The architecture of the CSPI200 shown in Figure 2 is merely an example and is not intended to be limiting. Alternative embodiments are possible, with variations, alternatives, and modifications. For example, in some implementations, the CSPI200 may include more or fewer systems or components than those shown in Figure 2, may combine two or more systems, or may include different configurations or arrangements of systems. The systems, subsystems, and other components shown in Figure 2 may be implemented using hardware, or a combination thereof, in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of each system. The software may be stored in non-temporary storage media (e.g., memory devices).

[0131] Figure 4 shows the connection between a host machine and an NVD to implement I / O virtualization to support multi-tenancy functionality in one embodiment. As shown in Figure 4, host machine 402 runs a hypervisor 404 that provides a virtual environment. Host machine 402 runs two virtual machine instances: VM1 406 belonging to customer / tenant 1 and VM2 408 belonging to customer / tenant 2. Host machine 402 has a physical NIC 410 connected to the NVD 412 via link 414. Each compute instance is connected to a VNIC run by the NVD 412. In the embodiment of Figure 4, VM1 406 is connected to VNIC-VM1 420 and VM2 408 is connected to VNIC-VM2 422.

[0132] As shown in Figure 4, NIC410 has two logical NICs: logical NIC A 416 and logical NIC B 418. Each virtual machine is connected to its own logical NIC and configured to work with that logical NIC. For example, VM1 406 is connected to logical NIC A 416, and VM2 408 is connected to logical NIC B 418. Even though the host machine 402 has only one physical NIC 410 shared by multiple tenants, each tenant's virtual machine is confident that it has its own host machine and NIC due to the logical NICs.

[0133] In one embodiment, each logical NIC is assigned its own VLAN ID. Thus, a specific VLAN ID is assigned to logical NIC A 416 for tenant 1, and another VLAN ID is assigned to logical NIC B 418 for tenant 2. When a packet is transmitted from VM1 406, the tag assigned to tenant 1 is attached to the packet by the hypervisor, and then the packet is transmitted from host machine 402 to NVD412 via link 414. In a similar manner, when a packet is transmitted from VM2 408, the tag assigned to tenant 2 is attached to the packet by the hypervisor, and then the packet is transmitted from host machine 402 to NVD412 via link 414. Thus, the packet 424 transmitted from host machine 402 to NVD412 has an associated tag 426 that identifies a specific tenant and associated VM. In NVD, for packet 424 received from host machine 402, the tag 426 associated with the packet is used to determine whether the packet should be processed by VNIC-VM1 420 or VNIC-VM2 422. The packet is then processed by the corresponding VNIC. The configuration shown in Figure 4 ensures that each tenant's compute instance owns its own host machine and NIC. The configuration shown in Figure 4 enables I / O virtualization to support multi-tenancy functionality.

[0134] Figure 5 shows a simplified block diagram of a physical network 500 according to one embodiment. The embodiment shown in Figure 5 is structured as a Clos network. A Clos network is a specific type of network topology designed to provide connectivity redundancy while maintaining high bimodal bandwidth and maximum resource utilization. A Clos network is a type of non-blocking multi-stage or multi-layer switching network, the number of stages or layers may be two, three, four, five, etc. The embodiment shown in Figure 5 is a three-layer network including layers 1, 2, and 3. A TOR switch 504 represents a layer 0 switch in the Clos network. One or more NVDs are connected to the TOR switch. A layer 0 switch is also called an edge device of the physical network. A layer 0 switch is connected to a layer 1 switch, also called a leaf switch. In the embodiment shown in Figure 5, a set of "n" layer 0 TOR switches is connected to a set of "n" layer 1 switches, together forming a pod. Each layer 0 switch in a pod is interconnected to all layer 1 switches in the pod, but there are no switch connections between pods. In one implementation, two pods are called a block. Each block is serviced by or connected to a set of n Layer 2 switches (sometimes called spine switches). Multiple blocks can exist in a physical network topology. The Layer 2 switches are then connected to n Layer 3 switches (sometimes called superspine switches). Packet communication over the physical network is typically performed using one or more Layer 3 communication protocols. Typically, all layers of the physical network except the TOR layer have n-way redundancy, thus enabling high availability. Policies may be specified for pods and blocks to control the visibility of switches to each other within the physical network, enabling scaling of the physical network.

[0135] A key feature of Clos networks is that the maximum number of hops required to reach a packet from one Layer 0 switch to another (or from one NVD connected to a Layer 0 switch to another NVD connected to a Layer 0 switch) is fixed. For example, in a Layer 3 Clos network, a packet may require a maximum of 7 hops to reach one NVD to another, with the source and target NVDs connected to the leaf layers of the Clos network. Similarly, in a Layer 4 Clos network, a packet may require a maximum of 9 hops to reach one NVD to another, with the source and target NVDs connected to the leaf layers of the Clos network. Therefore, the Clos network architecture maintains consistent latency throughout the network, which is crucial for communication within and between data centers. Clos topologies scale horizontally and are cost-effective. The network's bandwidth / throughput capacity can be easily increased by adding more switches (e.g., more leaf and spine switches) to different layers and by increasing the number of links between switches in adjacent layers.

[0136] In one embodiment, each resource within the CSPI is assigned a unique identifier called a Cloud Identifier (CID). This identifier is included as part of the resource information and can be used to manage the resource, for example, via the console or via an API. An example syntax for a CID is as follows: ocid1.<Resource Type>.<Realm>.[Region][.Future Use].<Unique ID> Here, ocid1: A literal string indicating the CID version. Resource type: The type of resource (e.g., instance, volume, VCN, subnet, user, group, etc.). Realm: The realm in which the resource resides. Examples of realm values ​​include "c1" for a commercial realm, "c2" for a government cloud realm, or "c3" for a federal government cloud realm. Each realm may have its own domain name. Region: The region where the resource resides. This section can be blank if the region cannot be applied to the resource. Future Use: Reserved for future use. Unique ID: The unique part of the ID. This format may vary depending on the type of resource or service.

[0137] Cloud Defense System This document describes techniques for monitoring and collecting data related to reverse lookup or recursive DNS (rDNS) traffic associated with monitored cloud environments.

[0138] A recursive domain name system (DNS) resolver can receive a fully qualified domain name (FQDN) from a requesting system as the subject of a DNS request, and in response to this request, it can perform a lookup of the corresponding IP address (for example, by using a trusted DNS resolver) and then return the IP address to the requesting system.

[0139] An rDNS request is a request that performs a lookup in the reverse direction compared to a DNS request. Therefore, upon receiving an rDNS request containing an IP address, a recursive DNS resolver can respond to this request by performing a lookup (e.g., by using a trusted DNS resolver) within a set of pointer (PTR) records for the corresponding fully qualified domain name (FQDN) using the received IP address and then returning it to the system requesting the FQDN. An rDNS request may be received from any system that wants to perform an FQDN lookup using an IP address. For example, a system may have received an IP address as part of one or more packets received from that IP address and therefore wants to know the FQDN associated with this IP address (e.g., to see if the IP address is trusted by trusting the FQDN). Another example is when something is trying to connect to a host machine using SSH; this something may generate a PTR query, as OpenSSH has several conditions that can cause a call to getaddrinfo which causes the host to issue a PTR query.

[0140] As another example, an rDNS request can help a system determine whether a second system (e.g., a server) has set up a PTR record, which can serve as a hint about the validity of packets received from the second system.

[0141] As another example, PTR records can be useful in determining where network traffic originates from within a system (for example, a domain name).

[0142] rDNS traffic may include recursive DNS (rDNS) requests originating from a monitored environment (e.g., a monitored cloud environment) and responses to those requests received from DNS resolvers. The collected data may then be analyzed to identify potential threats to the monitored environment. Potential threats may include not only actual threats but also irregular behavior in the monitored cloud environment. The collected data may be analyzed to identify potential sources of monitored behavior (e.g., threats) and to identify one or more parts of the monitored environment (e.g., threat targets) that are receiving the monitored behavior.

[0143] Figure 6 shows a simplified flowchart of the systems and processes involved in reverse DNS (rDNS) requests and rDNS responses in some embodiments.

[0144] In 610, source 602 (e.g., packet sender) may send a message (e.g., one or more packets) to target 604. The packets may contain data about source 602, such as the IP address of source 602 that sent the packets. For example, the IP address of source 602 may be included in the packet header. After target 604 receives the message or part of it from source 602, target 604 may perform an rDNS request to determine the domain name associated with the IP address of source 602.

[0145] 612 indicates that the rDNS request is sent to DNS resolver 608. DNS resolver 608 may be able to determine the domain name associated with the IP address by performing a lookup. If DNS resolver 608 has information about the domain name associated with the IP address, it may be able to return the domain name in response to the request that contained the IP address. If DNS resolver 608 does not have a domain name associated with the IP address, the rDNS request may fail.

[0146] At 614, DNS resolver 608 may send a response to the rDNS request. This response may include the domain name of source 602 (e.g., a fully qualified domain name (FQDN) obtained using the IP address associated with source 602). This response may be sent to target 604, which originally made the rDNS request.

[0147] Embodiments included herein may use a traffic monitoring system 606 to monitor rDNS requests sent by one or more targets 604 to one or more DNS resolvers 608. Furthermore, the traffic monitoring system 606 may be configured to monitor rDNS responses sent from one or more DNS resolvers 608 to one or more targets 604 in response to rDNS requests. The traffic monitoring system 606 may be able to obtain data from rDNS requests and responses such as timestamp (TS), message (e.g., port of source 602, fully qualified domain name of source 602, response) (Msg), IP address of source 602, IP address of target 604 (Src), query length of fully qualified domain name (qLen), query type (qType), response code (rc), time-to-live (ttl) value, value used to map the query back to the virtual network (VCNTSig), internal nameserver view (e.g., viewHash) used for resolution (e.g., namespaces routed by the internal internet and / or non-internet), flags to track whether the resolution path went through the internet or local (path), and / or other data contained in the packet sent between target 604 and DNS resolver 608. The data that can be obtained by the traffic monitoring system 606 may be called raw data.

[0148] Examples of the format of raw data and fields contained in raw data may be as follows: {"ts":1620002030957,"msg":"34157 90.73.0.10.in-addr.arpa.: no-answer","src":"10.9.90.17","qLen":24,"qtype":"PTR","rc":3,"ttl":10,"vcnTsig":"jbrjswk3sjyudsynektrvq==.","viewHash":"qw5guonijczl6nzkpsvmhmnpfe.view.alt.","path":"p"} or {"ts":1620002030963,"msg":"43377 212.47.115.62.in-addr.arpa.: [PTR las-b24-link.ip.twelve99.net.]","src":"10.7.108.21","qLen":27,"qtype":"PTR","rc":0,"ttl":3021,"vcnTsig":"x9elmuu0rectetf4cmpizq==.","path":"i"}.

[0149] The traffic monitoring system 606 may be included in the cloud defense system. The cloud defense system is described in more detail below.

[0150] Figure 7 shows a simplified flowchart of the systems and processes involved in obtaining monitored traffic data from rDNS requests and rDNS responses, according to some embodiments. System 700 illustrates how the systems and methods described with respect to System 600 can be used and scaled in a cloud computing environment.

[0151] System 700 illustrates how multiple sources (e.g., source A702a and source B702b) can each send one or more packets to one or more host machines (e.g., host machine A706a) via a VCN (e.g., VCN A704a). For example, source (e.g., source A702a) can send one or more packets to one or more host machines (e.g., host machine A706a). Furthermore, one or more host machines (e.g., host machine A) can receive one or more packets from one or more sources (e.g., source A702a).

[0152] Packets can be sent to host machines via each other for various reasons, such as when an email is being sent from source A702a to host machine A706a. In another example, when source B702b sends an openSSH command to host machine B706b, the packet is sent to host machine B706b.

[0153] In some embodiments, a VCN comprises one or more host machines. System 700 is shown to have one or more host machines. In detail, the shown VCN A704a comprises any number of other host machines, such as those represented by host machine A706a, host machine B706b, and host machine N706n. As further described herein, in some embodiments, one or more regions may comprise one or more VCNs.

[0154] After host machine A706a receives one or more packets from source A702a, host machine A706a may perform an rDNS request using DNS resolver A608a via VCN A704a. As described with respect to system 600, system 700 is shown to be configured to obtain traffic data from rDNS requests and responses between host machines and DNS resolvers using cloud defense system 708 (which may include traffic monitoring system 606). Requests and responses sent between host machines may be sent and received via the corresponding VCNs. Thus, the raw data described above with respect to system 600 may be obtained using cloud defense system 708. The monitored raw traffic data may be collected and stored as raw data 710.

[0155] Furthermore, any number of rDNS requests made by host machine B706b or any other host machine to any one of the multiple DNS resolvers can be monitored by the cloud defense system 708. Responses corresponding to rDNS requests can also be monitored by the cloud defense system 708.

[0156] Raw data 710 may be collected and stored by the cloud defense system 708 as a result of the cloud defense system 708 collecting monitored traffic data from one or more rDNS requests and / or responses between one or more host machines and one or more DNS resolvers. The raw data 710 may be used by the cloud defense system 708 to generate enhanced data 712.

[0157] Figure 8 shows a simplified architecture diagram of a system that communicates with and constitutes the cloud defense system 708, according to one embodiment.

[0158] System 800 shows a traffic monitoring system 606 configured to monitor traffic for rDNS requests and / or responses between the Cloud Service Provider Infrastructure (CSPI) 802 and one or more DNS resolvers. The components may be logical components, physical components, or a combination thereof.

[0159] As shown in System 700, rDNS requests may be sent from a VCN to a DNS resolver. The cloud defense system 708 may monitor rDNS requests and / or responses sent between one or more VCNs and one or more DNS resolvers via the traffic monitoring system 606, as already described. System 800 shows that in some embodiments, the cloud service infrastructure may comprise one or more regions, each of which may comprise one or more VCNs.

[0160] The traffic monitoring system 606 may be configured to monitor traffic in an environment (e.g., a distributed environment). The monitored environment may include one or more regions. The monitored environment may include one or more VCNs. Furthermore, each region may include one or more VCNs. In one embodiment, the monitored environment is part of one or more regions. In one embodiment, multiple traffic monitoring systems 606 may be used with the CSPI 802 to monitor different environments, the same environment, or different parts of the CSPI 802. Thus, each traffic monitoring system 606 of the cloud defense system 708 may be configured to monitor at least a portion of the CSPI 802. In one embodiment, a first traffic monitoring system 606 monitors the environment included in the first CSPI 802, and a second traffic monitoring system 606 monitors the environment included in the second CSPI 802.

[0161] As an example of how the traffic monitoring system 606 may be configured, it may be configured to monitor rDNS requests and / or responses sent between VCN A704a and one or more DNS resolvers. Thus, the raw data 710 collected from the traffic monitoring system 606 may include rDNS request and / or response data known to be associated with a particular VCN (e.g., VCN A704a).

[0162] As another example, the traffic monitoring system 606 may be configured to monitor rDNS requests and / or responses transmitted between regions (e.g., between at least some of the VCNs within a region). Thus, the raw data 710 collected from the traffic monitoring system 606 may include rDNS request and / or response data known to be associated with a particular region (e.g., region A804a) and at least some of the VCNs within that region.

[0163] In some embodiments, the traffic monitoring system 606 is configured to monitor rDNS requests and / or responses made at a global (e.g., all regions) level. Thus, the raw data 710 collected from the traffic monitoring system 606 may include rDNS request and / or response data known to be associated with a particular set of regions and / or VCNs (e.g., VCNs for regions A804a to N804n).

[0164] Furthermore, one or more DNS resolvers that receive one or more rDNS requests from one or more VCNs, regions, and / or CSPI802, etc., may, in some embodiments, be located within the environment where the cloud infrastructure resides (e.g., DNS resolver C708c). In some embodiments, one or more of the DNS resolvers may not be local to the network in which CSPI802 operates internally (e.g., DNS resolver A608a, DNS resolver B608b, DNS resolver N708n). System 800 indicates that there may be any number of DNS resolvers that can receive rDNS requests from VCNs (either within or outside CSPI802), and the VCNs may be located within one or more regions and / or CSPI802.

[0165] As shown in systems 600 and 700, raw data 710 is obtained by monitoring rDNS requests and / or responses transmitted between CSPI 802 (e.g., VCN A704a, VCN B704b, and / or region B804b). Raw data 710 may be stored in memory store 808. The data that may be contained within raw data 710 has already been described above with respect to system 600.

[0166] As an addition to or alternative to raw data 710 being obtained by the traffic monitoring system 606, raw data 710 may be obtained from an external data source 810. For example, an external data source 810 may be a data source containing data on network activity, IP address owners, fully qualified domain name owners, known scanners, and information on known attack methods (for example, an attack may cause a host to request a specific record, trigger a vulnerability in a recursive resolver within a VCN or CSPI, or extract additional metadata about the environment). An external data source 810 may also contain data obtained from other traffic monitoring systems 606 and / or other cloud defense systems 708. In an exemplary attack activity that may be recorded in an external data source 810, an attacker defines a PTR of an IP performing a brute-force attack, containing specific data or structures, for the purpose of enumerating details about the DNS infrastructure using attempted connections.

[0167] The external data source 810 may provide information to be used as raw data 710 or information to be combined with raw data 710. For example, the raw data 710 may include a first fully qualified domain name associated with a first rDNS response, and the external data source 810 may indicate that the fully qualified domain name is a known scanner (e.g., a scanner that should not be blocked, a scanner that should be blocked), and thus the external data source 810 may be used to complement the operation of the data augmentation system 812. Thus, in one embodiment, the external data source 810 can assist the data augmentation system 812 in sending data to the data augmentation system 812 and creating augmented data 712.

[0168] The data augmentation system 812 may be capable of generating augmented data 712 using the raw data 710. In one embodiment, the data augmentation system 812 may, in addition or alternatively, use an external data source 810 to generate augmented data 712 (e.g., a database of regional internet registrar information, or a portion of the augmented data 712 or a portion thereof). In one embodiment, the augmented data 712 includes at least the raw data 710.

[0169] Extended data 712 may include data from raw data 710. For example, extended data 712 may include a timestamp (TS), message (e.g., source port, source fully qualified domain name, response) (Msg), source IP address, target IP address (Src), query length for the fully qualified domain name (qLen), query type (qType), response code (rc), time-to-live (ttl) value, value used to map the query back to a virtual network (VCNTSig), internal name server view used for resolution (e.g., namespaces routed by the internal internet and / or non-internet) (viewHash), flags to track whether the resolution path went through the internet or local (path), and / or other data contained in the packet sent between the target and the DNS resolver.

[0170] Therefore, information including extended data 712 may be further obtained by using registrar (e.g., regional internet registrar) data. Registrar data may be used to identify the owner of an IP address, the range of IP addresses within NetRange, the organization, and / or other relevant details. For example, PTR or pointer records may provide a reverse mapping between IP addresses and domain names. A PTR may be the reverse of a PTR record, and the domain name to which the PTR record is mapped may provide context regarding interactions with IP addresses or may be used to associate IP addresses with a larger infrastructure.

[0171] Furthermore, routing table (e.g., RIPE, routeviews) data can be used to identify who is routing prefixes associated with IP addresses. In some embodiments, an OriginAS value is defined for prefixes in regional Internet registry (RIR) data, but if it does not match the routing data, a flag is associated with the prefix associated with the IP address. This flag may indicate that the data is irregular.

[0172] In one embodiment, the extended data 712 may include the inverted sender IP address, fully qualified domain name (FQDN), network identifier (e.g., TSIG), region, rDNS response code, network source from which network traffic originates (e.g., autonomous system number (ASN) ("OriginAS")), owner of the network from which network traffic originates (e.g., prefix owner ("Organization")), NetRange, CIDR, NetName, NetHandle, Parent, NetType, RegDate, Updated, Address, City, StateProv, Postal Code, Country, OrgAbuseHandle, OrgAbuseName, OrgAbusePhone, OrgAbuseEMail, OrgID, ASNumber, ASName, ASHandle, and / or other information associated with the sender IP address that is the subject of the rDNS request.

[0173] Therefore, each rDNS request can trigger the generation of raw data 710, which can then be used to generate extended data 712. The data extension system 812 can then send the generated extended data 712 to the memory store 808.

[0174] The augmented data 712 may be used to perform data analysis using the data analysis system 818. In one embodiment, the augmented data 712 may be clustered by the data augmentation system 812 to facilitate the data analysis performed by the data analysis system 818. Thus, the augmented data 712 may include clustered augmented data and / or unclustered augmented data.

[0175] Clustering can be performed to group similar augmented data together. For example, augmented data may be similar if the IP addresses are the same (e.g., source IP, target IP, IP looked up using rDNS requests), the TSig is the same (each VCN may have a unique VCN TSig), the prefix is ​​the same, the autonomous system is the same, etc. Therefore, clustering can group observations related to pointers. For example, clusters may group augmented data by the namespace of pointers to help identify clusters of infrastructure. Identifying clusters of infrastructure is possible by examining the entire augmented data to determine related infrastructure patterns that may arise from different prefixes or autonomous systems. In some embodiments, the false positive rate can be reduced by using a public suffix list.

[0176] For example, the IP addresses 209.141.58.151, 198.98.59.197, 209.141.35.27, and 199.195.254.209 can be mapped to the namespaces exit01.oxds(.)org, exit03.oxds(.)org, exit10.oxds(.)org, and exit17.oxds(.)org, respectively. Therefore, the extension data associated with each of the IP addresses in the above example can be grouped into one related cluster of namespaces all associated with the oxds(.)org namespace.

[0177] As a result of clustering, the total number of queries per region, per CSPI802, per VCN, per sender IP, per IP in pointer, and / or per VCN network identifier (e.g., TSIG) can be determined. By grouping similar information together, the scale of information can be used to determine trends (e.g., anomalies, patterns) in the augmented data 712. By analyzing the augmented data clusters, an analysis of the relevant information can be performed, providing further information about individual rDNS requests and / or responses compared to unclustered augmented data. The results of clustering can also lead to an understanding of whether individual nodes are being used to initiate network traffic or whether a superset of the infrastructure is being used. Furthermore, clustering augmented data can determine a larger group of nodes associated with a namespace.

[0178] The augmented data 712 (clustered augmented data and / or non-clustered augmented data) may be sent to the alerting system 816. The alerting system 816 may use the augmented data 712 to determine whether an alert should be generated based on the received augmented data 712. The alerting system 816 may be an existing alerting system 816 that can use the augmented data 712 obtained from the traffic monitoring system 606 and the data augmentation system 812 to generate alerts for one or more systems (e.g., a cloud defense system or another system). An alert may trigger the generation of a report (e.g., a log, email, printout), the occurrence of a configuration (e.g., setting a flag), or the execution of another action. In one embodiment, the alerting system may be capable of communicating with the cloud defense system to generate an alert and / or responding to the generation of an alert.

[0179] The extended data 712 (e.g., clustered extended data and / or non-clustered extended data) may be transmitted to the data analysis system 818. The data analysis system 818 may analyze the extended data 712 to determine trends, patterns, and anomalies, generate reports, respond to queries, perform system configurations, determine where network traffic is being generated, determine which baseline network traffic is observed and / or expected, and so on. The data analysis system 818 may comprise an analyzer subsystem 820, a report generation system 822, and / or a query system 824.

[0180] The analyzer subsystem 820 may be capable of analyzing the extended data 712. The analyzer subsystem 820 may be capable of generating one or more warnings or one or more reports. Furthermore, the analyzer subsystem 820 may be capable of identifying one or more patterns in the extended data 712. The identified patterns may represent baseline network activity or irregular network activity.

[0181] For example, baseline network activity may represent one or more P addresses, VCNs, regions, CSPI802s, networks, fully qualified domain names, and / or IP address owners that are sending and / or receiving requests (e.g., inside or outside of a CSPI802). Baseline network activity may represent the average number of rDNS requests made by a given VCN, region, CSPI802, etc., over a period (e.g., seconds, minutes, hours, days, weeks, months, years, etc.), time of day, or period. Baseline network activity may represent the fully qualified domain names and / or IP addresses that are the target of rDNS requests over a period. Baseline network activity may represent a baseline of any combination of measurements included in the extended data 712.

[0182] Baseline network activity can be determined using the traffic monitoring system 606, and may be determined using an average value (e.g., average number of packets from a first IP address) or other indicator that represents a previous trend related to any portion of the extended data 712 observed by the traffic monitoring system 606.

[0183] Baseline network activity can be used to determine expected network traffic in a given context (e.g., time). Baseline network activity can be used to establish a baseline for the number of rDNS resolver requests and / or responses sent and / or received by a monitored environment (e.g., VCN, region). In one embodiment, baseline network activity can be used to determine the number of rDNS requests containing the same IP address within a given period. In one embodiment, baseline network activity can be used to determine the number of responses for rDNS requests containing the same FQDN or associated with the same owner within a given period.

[0184] Baseline network activity may be used to compare with subsequently monitored network activity in the same or different monitored environments, to help determine which environment or part of an environment is responsible for irregular network activity, and / or to help determine the cause of the irregular network activity. Therefore, baseline network activity may be used to set network activity thresholds, allowing activity outside of these thresholds to be classified and / or further analyzed.

[0185] Observing network activity in a monitored environment and establishing a baseline network activity for that environment can reduce false positives of anomaly network activity, because observations can be compared to baseline network activity in various ways (such as comparing to network activity in the monitored environment, comparing to network activity in a subset of the monitored environment, or considering other contexts available within the augmented data). Therefore, obtaining baseline network activity can be used to more accurately identify what constitutes anomaly network activity.

[0186] For example, in one embodiment, if a monitored environment (e.g., a VCN, a region) generates multiple rDNS requests during a first period, and the number of such requests is significantly greater than the baseline rDNS requests during a second period (e.g., 50% more), the monitored environment may be considered under attack. The first and second periods may be the same length of time (e.g., one minute) and / or the same time frame (e.g., noon to midnight on the first day is compared to noon to midnight on the second day).

[0187] In one embodiment, the current network activity of a monitored environment may be compared to the corresponding baseline network activity metrics of the same or different monitored environments (e.g., VCNs, regions, and / or other monitored environments).

[0188] In one embodiment, if the network activity of the monitored environment differs sufficiently from the baseline network activity, any number of actions (e.g., zero or more) may be performed.

[0189] Actions that may be taken in response to the detection of network activity that differs significantly from baseline network activity may include determining at least one of the following: the portion of the monitored environment that the network activity is targeting (e.g., region A804a, VCN A704a) and the source from which the network activity is originating (e.g., an IP address, the owner of the IP address). In some embodiments, other information related to the irregular network activity, such as the port associated with the network activity, may also be collected.

[0190] The data analysis system 818 may also include a report generation system 822. The report generation system 822 may be capable of generating a report based on at least one of the analyses performed by the analyzer subsystem 820, the augmented data 712, and the inputs from the query system 824.

[0191] The report generation system 822 may be capable of generating reports containing specific information. The information contained in the generated report may depend on the type of report generated, what caused the report to be generated, the parameters used when initiating the report generation, and / or the data contained within the report.

[0192] For example, when the analyzer subsystem 820 aggregates network activity in the monitored environment (e.g., baseline activity, irregular activity, etc.), the report generation system 822 may cause a report to be generated by the report generation system 822, which may include at least a portion of the extended data 712 (e.g., data included in rDNS requests and responses).

[0193] The query system 824 may cause the report generation system 822 to generate a report when a user of the query system 824 submits a request to the query system 824 to obtain or generate a report. The user of the query system 824 may have the ability to specify the data they wish to be included in the generated report. In some embodiments, the user may be restricted (e.g., by user privileges) to the data that the user can request from the query generation system, and thereby to the data that may be included in the generated report.

[0194] The report generation system 822 may also monitor the extended data 712 to determine whether a report should be generated. In some embodiments, reports may be generated when certain conditions occur (e.g., an irregular event occurs, a request is submitted) or may be scheduled (e.g., reports are generated once a day).

[0195] The report generation system 822 may be capable of transmitting the generated report. The generated report may be transmitted to another system (e.g., a user device, a printer, another system, etc.).

[0196] The data analysis system 818 may also include a query system 824. The query system 824 may be capable of receiving queries generated by other systems and / or users. Queries received by the query system 824 may be received from users and / or systems within the environment of the cloud defense system 708 (e.g., the administrator of the cloud defense system 708, user B826), as well as from users and / or systems outside the cloud defense system 708 (e.g., public users, customers of the cloud service provider, other systems that can use the enhanced data 712, user A828, etc.). The query system 824 may be capable of receiving query inputs, performing a lookup of the enhanced data 712, and generating output in response to the received query.

[0197] In some embodiments, the inquiry system 824 is configured to send information to the system or user based on whether the system or user is within the cloud defense system 708 or has another method for determining the level of access.

[0198] In one embodiment, other systems may be configured to use the query system 824 to retrieve enhanced data 712 obtained from the cloud defense system 708, so that those systems can perform further actions. In one embodiment, the warning system 816 may be configured to use the query system 824.

[0199] Figure 9 shows a simplified flowchart 900 illustrating a method for protecting a monitored environment using rDNS traffic, according to one embodiment. In the exemplary embodiment shown in Figure 9, the processes shown in flowchart 900 may be performed by the cloud defense system 708.

[0200] In 902, information is received that identifies the environment to be monitored (e.g., a distributed environment). The monitored environment may include a portion of the infrastructure provided by the CSP to deliver one or more cloud services to the CSP's customers. The monitored cloud environment may include one or more VCNs that run payloads for the CSP's customers. For example, the monitored environment may be the CSP's data center, a portion of a data center, multiple data centers within a region, infrastructure across multiple regions, global infrastructure, etc. In one embodiment, the information received in 902 may identify one or more VCNs to be monitored (e.g., unique network identifiers of the VCNs (VCN TSigs)), for example, one or more VCNs belonging to one or more customers (one or more VCNs may be associated with one or more customers, regions, etc.).

[0201] In 904, rDNS traffic related to the monitored environment identified in 902 is monitored over a period of time. Monitoring rDNS traffic in 904 may include monitoring requests originating from the monitored environment and / or monitoring responses to rDNS requests directed to the monitored environment and received from one or more DNS resolvers. As previously mentioned with respect to flow 600 in Figure 6, an rDNS request generally results in the reception of a corresponding rDNS response from a DNS resolver, but in some circumstances, the DNS resolver may not produce a response. This can happen, for example, if the DNS resolver does not have a pointer record stored for the IP address identified in the rDNS request.

[0202] In 906, data is collected and stored regarding the monitored environment related to the monitored rDNS traffic, based on the monitoring performed in 904. For the purposes of this disclosure, the data collected and stored in 906 may be referred to as raw data. This is to distinguish it from the enhanced data described below. The term raw data is not intended to limit in any way the scope of the claimed embodiments. The data that may be included in rDNS requests and / or responses, as well as the data that may be collected and stored in 906, are described above with respect to flow 600 in Figure 6.

[0203] Raw data can be stored in a variety of different formats, including documents, tables, files, data stores, data lakes, and databases. Raw data can be stored locally and / or remotely from cloud-based security systems.

[0204] In 908, the raw data collected and stored in 906 is augmented with additional information to generate augmented data. There are various ways in which raw data can be augmented. For example, augmentation may include adding additional information to the raw data, or supplementing the raw data with additional information, organizing the raw data in a desirable way to facilitate analysis (e.g., clustering the data along one or more dimensions), or replacing at least a portion of the raw data.

[0205] The raw data may include a timestamp (TS), message (e.g., port of source 602, fully qualified domain name of source 602, response) (Msg), IP address of source 602, IP address of target 604 (Src), query length of the fully qualified domain name (qLen), query type (qType), response code (rc), time-to-live (ttl) value, value used to map the query back to the virtual network (VCN TSig), internal name server view used for resolution (e.g., viewHash) (e.g., namespaces routed by the internal internet and / or non-internet), flags to track whether the resolution path went through the internet or local (path), and / or other information contained in the packet sent between target 604 and DNS resolver 608.

[0206] Information added to the raw data may be obtained from various sources, including sources within the monitored environment and sources outside the monitored environment. Sources outside the monitored environment may include sources provided by the CSP or sources provided by other third parties. For example, additional raw data (e.g., raw data collected by another monitored environment, raw data collected at different times) may be added to the raw data to generate augmented data or a portion of augmented data. In another example, augmented data (e.g., augmented data collected by another monitored environment, augmented data collected by the same monitored environment at different times) may be added to the raw data.

[0207] Complementing raw data may involve using the raw data to obtain other data that can be included in the extension. For example, extension data may include raw data or a portion of raw data. Extension data may include data obtained using registrar (e.g., regional internet registrar) data to complement the raw data. Registrar data may be used to identify the owner of an IP address, the range of IP addresses within NetRange, the organization, and / or other relevant details. For example, PTR or pointer records from raw data may provide a reverse mapping between IP addresses and domain names. A PTR may be the reverse of a PTR record, and the domain name to which the PTR record is mapped may provide context regarding the interaction with the IP address or may be used to associate the IP address with a larger infrastructure.

[0208] Furthermore, when supplementing raw data to generate enhanced data, routing table data (e.g., RIPE, routeviews) may be used to identify who is routing the prefixes associated with the IP addresses contained in the raw data. In some embodiments, an OriginAS value is defined for prefixes in Regional Internet Registry (RIR) data, but if it does not match the routing data, a flag is associated with the prefix associated with the IP address. This flag may indicate that the data is irregular.

[0209] Enhanced data can be generated by replacing at least a portion of the raw data. Data fields contained in the raw data can be replaced with known relevant information. For example, the raw data may contain a first fully qualified domain name (FQDN), and the cloud defense system may be configured to replace the ".com" portion of the FQDN with a value representing the top-level domain (TLD) characteristics (e.g., ".com"). Such value replacement may reduce processing time. Furthermore, to maintain the privacy of one or more values ​​in the enhanced data, the values ​​can also be replaced with encrypted values.

[0210] Enhanced data can be generated by deleting at least a portion of the raw data. For example, lifetime values ​​may be removed when they are deemed not to provide valuable insights into the monitored environment. Removing such information can reduce the amount of storage and time used to process enhanced data, allowing irregular activity to be recognized more quickly and corresponding actions to be taken sooner. In some embodiments, deleting at least a portion of the enhanced data may be done to enhance privacy. For example, a customer using a monitored environment may not want one of the monitored network activities of the VCN to be used to create enhanced data, or may want the VCN TSig to be removed from the raw data to create enhanced data.

[0211] Enhanced data can be generated by organizing at least a portion of the raw data. Organizing raw data can be done to improve the efficiency of further analyses. For example, raw data can be organized at a hierarchical level. For instance, enhanced data can be generated by organizing all rDNS requests from a first VCN into a single enhanced data entry. In another example, enhanced data can be generated by organizing all rDNS requests from a first region (e.g., a first VCN and a second VCN) into a single enhanced data entry. In yet another example, enhanced data can be generated by organizing responses to all rDNS requests containing the same FQDN into a single enhanced data entry. Thus, organizing raw and / or enhanced data can be performed across multiple dimensions and is sometimes referred to as "clustering."

[0212] In one embodiment, the extended data may include the inverted sender IP address, fully qualified domain name (FQDN), network identifier (e.g., VCN TSIG), region, rDNS response code, network source from which network traffic originates (e.g., Autonomous System Number (ASN) ("OriginAS")), owner of the network from which network traffic originates (e.g., prefix owner ("Organization")), NetRange, CIDR, NetName, NetHandle, Parent, NetType, RegDate, Updated, Address, City, StateProv, Postal Code, Country, OrgAbuseHandle, OrgAbuseName, OrgAbusePhone, OrgAbuseEMail, OrgID, ASNumber, ASName, ASHandle, and / or other information associated with the sender IP address that is the subject of the rDNS request.

[0213] Augmented data may be used to determine the baseline network activity of a monitored environment or a part thereof. In the first example, augmented data may be used to determine the baseline rDNS requests sent by the monitored environment or a part thereof (e.g., a VCN). In the second example, augmented data may be used to determine whether the monitored environment is observing activity different from the baseline network activity of the monitored environment. The baseline network activity of the monitored environment may be based on previous augmented data obtained from the same or different monitored environments, may be configured by the user of the cloud defense system, and / or may be based on one or more external data sources.

[0214] In 910, the augmented data is used to identify irregular network activity related to the monitored environment. In one embodiment, the processing performed in 909 may include performing the processing in 912 and 914.

[0215] In 912, the augmented data generated in 908 is used to identify components of the monitored environment that may be targets of malicious activity. The augmented data can also be used to determine whether the monitored environment is observing behavior different from baseline network activity. Monitored network activity may differ from the baseline network activity of the monitored environment if the differential threshold is exceeded. The differential threshold can be set using configurations made by the cloud defense system user and / or monitored baseline data.

[0216] Some examples of thresholds that can be evaluated include the number of rDNS requests made by a particular monitored environment (e.g., VCN, region) over a specific period, the number of rDNS requests containing a first piece of information (e.g., IP address), the number of responses to rDNS requests containing a second piece of information (e.g., fully qualified domain name), the number of rDNS requests within a cluster of augmented data, the number of rDNS requests determined to be associated with a particular IP address owner, the time between one or more rDNS requests containing the same information, etc. This threshold may relate to one or more fields of information from the augmented data. In one embodiment, whether the baseline is sufficiently different from the augmented data may be determined using a machine learning model (e.g., to determine if the similarity between the baseline and the augmented data is within the threshold).

[0217] Another example of a threshold that can be evaluated is the number of rDNS requests that do not return a response, and as a two-stage analysis, the number of FQDNs returned for NXDomain / ServFail and the number of appropriate resolutions for those FQDNs. Another example of a threshold that can be evaluated is the number of rDNS requests for paired IPv4 and IPv6 addresses on a dual-stack host, which can be used to establish or fix coordination between v4 and v6 prefixes. Thresholds can be used to evaluate common infrastructure, such as trusted DNS providers, for a portion of the in-addr.arpa zone.

[0218] Therefore, augmented data obtained from the monitored environment may be compared to monitored baseline network activity and / or user-selected baseline network activity to determine whether one or more parts of the monitored environment (e.g., VCN, region) may be targeted by a malicious attack and / or whether network activity that could be considered unexpected (e.g., based on user-defined expectations and / or historical network activity) is occurring, and thus may indicate potential malicious activity.

[0219] In section 914, extended data may be used to identify the source of malicious activity targeting a monitored environment. The source may be identified by determining the source (e.g., IP address, fully qualified domain name, owner) corresponding to extended data that exceeds a defined baseline threshold.

[0220] In step 916, a signal may be output indicating irregular network activity identified during step 910. This signal may be output to a component of the cloud defense system and / or to another system. One system that may be outside the cloud defense system may be an alert system used to generate and transmit alerts based on output signals from one or more systems. A signal may be generated and output during step 916 regardless of whether irregular network activity is identified during step 910. Thus, this signal may indicate whether or not irregular network activity is occurring. Furthermore, regardless of whether or not irregular network activity is identified during step 910, the flow shown by flowchart 900 may be repeated any number of times. For example, the cloud defense system may continuously perform processes to continuously monitor the environment for irregular network activity. Thus, additional collected raw data may be augmented and combined with existing augmented data for further analysis.

[0221] In 918, one or more actions may be initiated in response to a signal output during 916. Any number of actions may be taken after one or more components of the monitored environment are identified as potentially under attack (for example, by determining that irregular network activity is occurring compared to a threshold of one or more baseline network activity) and / or after one or more sources of potentially malicious activity are identified in 914. For example, system configuration may be performed (e.g., taking the VCN offline, blocking an IP address, dropping a packet, configuring a firewall, generating logs, generating reports, patching the system, rerouting network traffic, contacting the source).

[0222] In an example where network traffic is rerouted, in one embodiment, network traffic may be rerouted to a highly interactive honeypot. Available data may allow the honeypot to match the specifications and services of the host on which it is deployed. The honeypot may allow the observer's actions once the connection is established and the actor has accessed the host.

[0223] Figure 10 shows a simplified flowchart for monitoring a monitored environment, determining baseline network activity, and deciding on actions to take when irregular activity is identified, according to some embodiments. In the exemplary embodiment shown in Figure 10, the process shown in flowchart 1000 may be performed by the cloud defense system 708.

[0224] In 1002, augmented data collected (for example, over a period of time) may be used to generate a baseline of the monitored environment. The augmented data may be collected from the same monitored environment between 904, 906, and 908, or in a similar manner to how augmented data was obtained from different environments. In some embodiments, baseline network activity is generated using an algorithm or a user-specified value.

[0225] Extended data may represent a baseline of the monitored environment or a part thereof. For example, extended data may represent that the monitored environment sends an average of 400 rDNS requests within a first period to resolve a first IP address. Thus, it can be determined that there are 400 baseline rDNS requests for that IP address, the corresponding FQDN identified in the possible rDNS request responses, the owner associated with the IP address, and / or the monitored environment within a period equal in length to the first period.

[0226] A baseline may be generated with respect to the extended data, including the IP address triggering the rDNS request, the FQDN associated with the IP address, the owner associated with the IP address, the type of request, and / or any other information. A baseline may also be generated to determine the number of rDNS requests being sent by the monitored environment or a part of it. Furthermore, a baseline may be specific to one or more VCNs, regions, customers, the entire monitored environment, etc.

[0227] A baseline can have a range, which is a portion (or all) of the monitored environment to which the baseline applies (for example, the baseline represents the network activity of VCN A and VCN B as a whole, or the baseline represents the network activity of VCN A and VCN B as an average). A baseline can also have a target, which is a component of the monitored environment to which the baseline is evaluated (for example, determining whether VCN A (the target) exceeds the baseline threshold). The target may include the same number or fewer system components as the monitored environment.

[0228] Therefore, whether the baseline has deviated can be evaluated against a single VCN in the monitored environment or against two or more VCNs in the monitored environment, depending on the target. In yet another example where the monitored environment includes a group of VCNs, a first baseline may be evaluated against a group of VCNs (targets) that fall within the baseline range (group of VCNs) in order to make a single decision on whether the baseline has deviated. In another example, a second baseline may be evaluated for each VCN (target) within the range (group of VCNs). Furthermore, each target in the monitored environment being compared against the baseline may have its own (e.g., unique and independently determined) baseline threshold compared to other targets in the monitored environment.

[0229] In 1004, additional monitoring may occur to determine whether the baseline is deviating, in a manner similar to how a baseline can be established by monitoring the monitored environment or a portion thereof. Thus, in 1004, augmented data collected with respect to the monitored environment is used to identify deviations from the baseline.

[0230] A deviation from the baseline may be determined based on the fact that the baseline value of one or more extended data information fields differs from the baseline value of one or more extended data information fields (e.g., lower, higher, or significantly further from a threshold).

[0231] For example, if the baseline number of rDNS requests sent by a monitored environment is 500 within one minute, the baseline may be set to 500 plus or minus 50 (or 10%). Therefore, if the monitored environment sends more than 550 rDNS requests or fewer than 450 rDNS requests, it may be flagged as a deviation. Similarly, the baseline may have varying levels of specificity, such as the number of responses to rDNS requests resolved by the monitored environment, the number of responses to rDNS requests resolved to a given FQDN, or how far apart sets of rDNS requests related to the resolution of all IP addresses or IP addresses owned by the same person are, or other fields of extended data may be compared.

[0232] In 1006, if a deviation is identified during 1004, this deviation may be flagged as an irregular activity related to the monitored environment. Depending on whether the activity is flagged, one or more actions may then occur.

[0233] In 1008, in response to the identification of the activity flagged in 1006, one or more actions may be performed. A warning may be one of the actions performed.

[0234] In 1008, a set of actions (e.g., a warning) may be identified. Each action in the set of actions may have one or more conditions associated with it. These conditions may relate to information contained in the extended data. Thus, if a certain value or combination of values ​​exists in the extended data, the conditional logic may be satisfied, triggering one or more actions. Furthermore, or alternatively, the actions (e.g., a warning) may be based on time (e.g., reporting data at a certain time interval).

[0235] Actions can relate to whether a particular IP address is being observed and / or whether a certain number of requests from a single IP address, FQDN, or owner are being observed. Furthermore, actions can relate to whether an attack is likely to occur, whether a microscan is detected, and / or whether a scan is detected.

[0236] In some embodiments, a DNS service update may occur as an action. For example, a two-phase PTR may return an FQDN (e.g., valid or invalid). If the FQDN is invalid and the IP belongs to an enterprise, this may indicate a DNS mismatch, triggering a DNS service update that removes the PTR as designated a conceptual maintenance activity. The action may also send an abuse report to other providers. As another example of an action, if multiple IPs are observed from the same prefix, ASN, or associated namespace, this may be an indication of a larger problem that necessitates alerting other systems (e.g., within and outside the network), and possibly other networks.

[0237] Actions may be composed of users of the cloud defense system (e.g., internal user B, external user A), users of another system, processes of another system, etc.

[0238] For each action in a series of actions, the conditions for the action may be evaluated to determine whether the conditions for the action are met and, therefore, whether the action should be performed. The decision may be made using the extended data obtained in 1004. This decision may evaluate whether one or more conditions associated with the action are met. This decision may evaluate whether there are any deviations flagged in 1006. A non-exhaustive example of such conditions might be whether each VCN from the monitored environment is sending more (e.g., twice as many, 1.5 times as many) or fewer rDNS requests than the baseline of the VCN and / or the monitored environment. Another condition might be whether each VCN from the monitored environment is sending more (e.g., 100 more) or fewer rDNS requests than the baseline of the VCN and / or the monitored environment by a defined value. Another condition might be whether each VCN from the monitored environment is sending multiple rDNS requests to look up a number of specific IP addresses far greater than the baseline (e.g., the baseline for a particular IP address), a number of specific IP addresses far greater than the baseline of the specific VCN making the request, and / or a number of specific IP addresses far greater than the baseline of the specific monitored environment making the request. These conditions may be the same or different conditions that were evaluated when determining whether irregular network activity occurred.

[0239] Furthermore, depending on the conditions that are met, different types of actions may be taken for the same irregularity, determined based on the severity of the irregularity (e.g., the difference from the baseline, the portion of the monitored environment where the irregularity occurs, etc.).

[0240] If the conditions for an action are not met, a further evaluation of the conditions for the action may occur. For example, additional supplementary data may then be collected to determine whether the conditions for the action have been met (for example, by returning to 1004).

[0241] If the conditions for an action are met, the corresponding action may be executed. One or more actions may be initiated as a result of one or more conditions for an action being met. A non-exclusive list of actions has already been described above. Some exemplary actions may include configuring a firewall, dropping packets, isolating a machine or part of a network, creating a report, sending a report, logging data, blocking an IP address, FQDN, and / or owner, or unblocking an IP address, FQDN, or owner (e.g., the IP address has not been seen for a period of time, or the number of packets received from the IP address has decreased again (e.g., below a threshold amount or allocation for network activity)).

[0242] Figure 11 shows a simplified flowchart for determining the baseline rDNS request volume of a monitored environment, according to some embodiments. In the exemplary embodiment shown in Figure 11, the process shown in flowchart 1100 may be performed by the cloud defense system 708. Flowchart 1100 may be an example of a process that occurs during 1002.

[0243] In 1102, a set of one or more VCNs that are the source of one or more rDNS requests are identified, and these VCNs may also be located within the monitored environment. The source of an rDNS request may be identified by determining the IP address used within the sender block of the rDNS request, or by using a unique network identifier (e.g., VCN TSig). The source of an rDNS request may be included in the extended data. The monitored environment may be any range of environments and therefore may include one or more VCNs, regions, customers, etc.

[0244] In 1104, a decision is made for each VCN identified in 1102, which evaluates rDNS information related to the VCN from the extended data. Information related to the VCN may include any amount of data that can be obtained from rDNS requests and / or responses. For example, the amount of rDNS requests and / or responses to rDNS requests may be determined from the extended data on a scale such as monitored systems, regions, and / or VCNs. In one example, further or alternatively, the amount of rDNS requests and / or responses to rDNS requests may be determined from the extended data with respect to a particular set of one or more IP addresses, FQDNs, and / or owners. In one example, the number of rDNS requests that did not return an FQDN may be determined. In yet another example, one or more fields of the extended data may be determined using rDNS information as described herein (for example, above).

[0245] As a first example of determining the volume of rDNS requests, the monitored environment may include VCN A, VCN B, and VCN C. During a first period, 1104 may determine that 400 rDNS requests originated from VCN A, 100 rDNS requests from VCN B, and 100 rDNS requests from VCN C.

[0246] In 1106, baseline network activity for the monitored environment may be determined based on the actions performed during 1104. The baseline may be generated for the entire monitored environment (e.g., one or more VCNs, one or more regions, one or more customers, etc.) or for a part of it (e.g., one VCN, two VCNs, one region, one customer, etc.).

[0247] The volume of rDNS requests originating from a VCN, the IP addresses identified in those requests, and / or other information associated with the IP addresses identified in the requests during 1104 can be aggregated with one or more other VCNs to determine a baseline for the entire environment or a portion thereof. Baseline network activity may include and / or be collected over a period of time that represents augmented data representative of a certain period. Baseline network activity may be a statistical representation of rDNS requests originating from the monitored environment. The statistical representation may represent the monitored environment as a whole, or a specific subset of rDNS requests originating from the monitored environment.

[0248] Using the first example from above, in 1104, the baseline network activity of a monitored environment may be determined to be 600 rDNS requests in one embodiment. In one embodiment, the baseline network activity of the first example may be finer in granularity, and it may be determined that VCN A makes an average of 400 rDNS requests, VCN B makes an average of 100 rDNS requests, and VCN C makes an average of 100 rDNS requests. Those skilled in the art who benefit from the present disclosure will recognize other ways in which a statistical baseline may be determined using augmented data acquired with respect to a monitored environment, using not only the number of rDNS requests but also any combination of fields of augmented data to form a particular baseline.

[0249] As described above and further detailed below, baseline network activity in a monitored environment or a portion of a monitored environment may be used to identify one or more components of the monitored environment where unexpected network activity is occurring (e.g., a potential target of malicious activity), and / or to identify the source of unexpected network activity (e.g., malicious activity) targeting at least a portion of the monitored environment.

[0250] The following is an example of how baseline network activity may be obtained for a range of VCNs, but similar techniques may be performed for other ranges. For each VCN identified in 1102, baseline network activity for each VCN may be determined. In some embodiments, baseline network activity is determined for a given VCN.

[0251] The extended data is used to determine the volume of rDNS requests originating from a single VCN and the IP addresses identified for lookups in those requests (e.g., IP addresses used to look up fully qualified domain names). The volume of rDNS requests originating from a single VCN can be determined by summing up the total number of rDNS requests generated by that single VCN. Summarizing rDNS requests can be done in various ways, such as summing up all rDNS requests, summing up all rDNS requests that identified a unique IP address for lookup, summing up all rDNS requests that identified a matching IP address for lookup, or summing up all rDNS requests based on other information contained in the extended data.

[0252] As a first example, during the first period, 400 rDNS requests are generated from VCN A to look up IP address A, 100 rDNS requests are generated from VCN A to look up IP address B, and 100 rDNS requests are generated from VCN A to look up IP address C.

[0253] The baseline network activity of VCN A can be determined during a first period. The baseline network activity may be a statistical representation of the rDNS requests originating from VCN A. This statistical representation may represent VCN A as a whole, or it may represent one or more specific subsets of the rDNS requests originating from VCN A.

[0254] Continuing from the first example above, the baseline for VCN A may, in one embodiment, be determined to be 600 rDNS requests. In one embodiment, the baseline may be determined to be that VCN A makes an average of 200 rDNS requests for any particular lookup of an IP address. In one embodiment, the baseline in the first example may be finer in granularity, and it may be determined that an average of 400 rDNS requests look up IP address A, an average of 100 rDNS requests look up IP address B, and an average of 100 rDNS requests look up IP address C. Those skilled in the art who benefit from this disclosure will recognize other methods by which statistical baselines may be determined using extended data obtained per VCN.

[0255] Similar baseline representations can be determined for any range of monitored environments (e.g., multiple VCNs, one region, multiple regions, etc.).

[0256] Figure 12 shows a simplified flowchart for detecting irregular network behavior in a monitored environment according to some embodiments. In the exemplary embodiment shown in Figure 12, the process shown in flowchart 1200 may be performed by the cloud defense system 708. Flowchart 1200 is an example of a baseline having a range of regions including one or more VCNs and targets of the VCNs in the monitored environment. However, embodiments that detect irregular activity over a changing range of the monitored environment, such as a monitored environment including two or more regions, two or more customers, etc., are also considered.

[0257] In 1202, extended data of the monitored environment is acquired over a period of time. This extended data can be acquired in a similar manner to, for example, those described in 902, 904, 906, and 908 above. For example, rDNS requests originating from a VCN within the monitored environment can be used to generate this extended data.

[0258] In step 1204, steps 1206, 1208, 1210, 1212, and 1214 may be performed for each VCN (target) within the monitored environment (scope). Such steps involve comparing extended data from targets in the monitored environment with baseline network activity in the monitored environment represented by baseline extended data of the monitored environment to determine whether one or more actions should be taken. An example of a step performed might be comparing extended data for a specific target VCN within the scope of the monitored environment with a baseline (e.g., taken from the monitored environment, a part of the monitored environment, or another monitored environment), where the monitored environment includes one or more VCNs (e.g., regions). Any number of baselines may be evaluated for each target, and any number of targets less than or equal to the number of system components included in the scope may be evaluated for one or more irregularities.

[0259] In 1206, the number of rDNS requests originating from each VCN over a certain period can be determined using augmented data collected from monitoring the environment between 1202 and 1206. This period may be the same period (e.g., the same amount of time, same time) or a similar period (e.g., 1.2 hours compared to 1 hour) observed when obtaining baseline augmented data for baseline network activity.

[0260] In 1208, the information determined during 1206 and the baseline of the monitored environment can be compared. Next, any portion of the data in the augmented data collected during 1202 that relates to each VCN of the monitored environment can be compared to the baseline of the monitored environment, a portion of the monitored environment (e.g., one VCN of the monitored environment, the same respective VCN).

[0261] At 1210, based on the comparisons performed between 1208 and 1210, a determination is made as to whether irregularities exist by comparing the monitored environment to the baseline network activity of the monitored environment.

[0262] An irregularity may be determined to exist if the baseline differs sufficiently from the augmented data obtained from at least a portion of the monitored environment. For example, the average number of rDNS requests for at least a portion of the monitored environment (e.g., for a particular VCN) may be compared to the baseline average number of rDNS requests for the environment's VCNs or the baseline for a particular VCN. Any of the baseline network activity augmented data collected from the processes performed in Figure 11 may be used to compare with the augmented data information determined in 1206 related to the monitored environment.

[0263] Furthermore, baseline network activity can be determined to be sufficiently different from the monitored environment using at least one of several evaluation criteria. One criterion might be whether each VCN from the monitored environment is sending more (e.g., twice as many, 1.5 times as many) or fewer rDNS requests than the baseline of the VCN and / or the monitored environment. Another criterion might be whether each VCN from the monitored environment is sending more (e.g., 100 more requests) or fewer rDNS requests than the baseline of the VCN and / or the monitored environment by a defined value. Another criterion might be whether each VCN from the monitored environment is sending multiple rDNS requests to look up a sufficiently large number of specific IP addresses than the baseline (e.g., the baseline for a particular IP address), a sufficiently large number of specific IP addresses than the baseline of the specific VCN making the request, and / or a sufficiently large number of specific IP addresses than the baseline of the specific monitored environment making the request. Another criterion might be the use of the standard deviation of values ​​in the acquired augmented data, which is then compared to the baseline of the corresponding values ​​in the monitored environment for that portion.

[0264] Various other conditions and combinations of conditions can be used to compare one or more values ​​from augmented data to a baseline, which can help determine which activities should be defined as irregular and provide insights into the activities occurring within the monitored environment.

[0265] Such conditions may make it possible to determine irregular behavior compared to a baseline, including, but not limited to, scans from new sources, drops in scans from sources, sources of denial-of-service attacks, and / or insider threats.

[0266] If no irregularity is detected between 1210 and 1212, monitoring of the monitored environment may continue by returning to 1202. If an irregularity is detected between 1210 and 1210, one or more actions may be initiated accordingly between 1214 and 1214.

[0267] If an anomaly is detected in the monitored environment during 1210, 1214 may be performed. In some embodiments, if it is detected that no anomaly exists, action may still be performed, such as by reducing restrictions on system access to the network. If an anomaly is detected, a signal identifying this anomaly may be output. This signal may be an email, text, or other transmission.

[0268] As a result of the output signal at 1214, any number of actions may be performed during 1216. In one embodiment, system configurations may occur, such as configuring a router to drop packets, configuring a firewall, configuring a VCN to drop packets, configuring a VCN to add to a blocklist, generating a report, logging at least some of the extended data from the monitored environment, sending communications, disconnecting the VCN from the network, or other similar actions, in order to prevent a known attacker from sending packets to the monitored environment or a network associated with the monitored environment.

[0269] Processes similar to those described in flowchart 1200 may also be applied to different targets and / or ranges of the monitored environment, thus enabling fine-grained control over the baseline, deviations from appropriate baseline thresholds, targets, ranges, and actions.

[0270] Figures 13A and 13B show a simplified flowchart for detecting a malicious actor using a cloud defense system, according to some embodiments. This simplified flowchart can be used to determine an actor acting in an unexpected (e.g., malicious) manner with respect to a VCN, region, one CSPI, or multiple CSPIs. In the exemplary embodiment shown in Figure 13, the process shown in flowchart 1300 may be performed by the cloud defense system 708.

[0271] In 1302, an IP address level threshold is determined for rDNS requests. The IP address level threshold can be based on a default (e.g., user-specified) IP address level threshold and / or a baseline value for rDNS requests. For example, the IP address level threshold could be 500 rDNS requests associated with individual IP addresses in the environment. The first IP address level threshold for a first IP address can be the same as or different from the second IP address level threshold for a second IP address.

[0272] In step 1304, the threshold for FQDN levels to be associated with an rDNS request is determined. The FQDN level threshold can be based on a default (e.g., user-specified) FQDN level threshold and / or a baseline threshold for FQDN levels of rDNS requests. For example, the FQDN level threshold could be 500 rDNS requests associated with a first FQDN in the environment. The first FQDN level threshold for the first FQDN can be the same as or different from the second FQDN level threshold for the second FQDN.

[0273] In step 1306, the owner level threshold for associating an rDNS request is determined. The owner threshold may be based on a default (e.g., user-specified) owner threshold and / or a baseline threshold for owners of rDNS requests. For example, the owner threshold may be 500 rDNS requests associated with the first owner of the environment. The first owner level threshold for the first owner may be the same as or different from the second owner level threshold for the second owner.

[0274] In step 1308, extended data about the monitored environment is obtained. Extended data can be obtained by monitoring the monitored environment over a period of time. The monitored data may reflect network traffic (e.g., of the monitored environment, of a portion of the monitored environment, or of different environments) over a period of time. Extended data can be obtained by using data obtained from rDNS requests and / or responses to rDNS requests.

[0275] In 1310, the extended data may be used to identify a set of one or more unique IP addresses involved in one or more rDNS requests. In some embodiments, the identified set of one or more IP addresses is a subset of the IP addresses involved in all rDNS requests. For example, this subset may be determined based on TSIG, VCN, region, etc. This may be because, in some embodiments, it may be desirable not to analyze certain rDNS requests (e.g., additional privacy is desired). The identified IP addresses may later be used to cluster the extended data associated with each IP address (e.g., to determine the number of rDNS requests associated with each IP address).

[0276] In 1312, using extended data (e.g., a set of one or more IP addresses), a set of one or more unique fully qualified domain names (FQDNs) involved in one or more rDNS requests may be identified (e.g., as a response to the indicated rDNS request). Each FQDN may be returned by a DNS resolver in response to a DNS request that contained the IP address associated with the FQDN. The identified FQDNs may then be used to cluster the extended data associated with each FQDN (e.g., to determine the number of rDNS requests associated with each FQDN).

[0277] At 1314, using extended data (e.g., a set of one or more IP addresses, a set of one or more unique FQDNs), a set of one or more owners involved in one or more rDNS requests may be identified (e.g., using a whois IP lookup, using an obtained database of IP owners, using an obtained database of FQDN owners). The identified one or more owners may then be used to cluster extended data associated with each owner (e.g., to determine the number of rDNS requests associated with each owner).

[0278] 1316 illustrates the connection between the process described in Figure 13A and the process described in Figure 13B.

[0279] At 1318, processes 1320, 1322, and 1322 may be performed for each unique IP address identified at 1310.

[0280] At 1320, using extended data associated with the unique IP address, the number of rDNS requests that include the unique IP address may be determined.

[0281] At 1322, a comparison between the number of rDNS requests that include the unique IP address (determined at 1320) and the IP-level threshold determined at 1302 may be performed. Accordingly, it may be determined whether the number of rDNS requests that include the unique IP address exceeds or falls below the IP-level threshold (e.g., by a certain amount).

[0282] At 1324, a determination is made as to whether the IP-level threshold for the unique IP address has been exceeded.

[0283] If the IP-level threshold is not exceeded, processing may continue by checking whether other thresholds (e.g., IP-level thresholds for different IP addresses, fully qualified domain name level thresholds, owner level thresholds) are exceeded, and / or by continuing to monitor the environment (e.g., by running 1308).

[0284] If the IP level threshold is exceeded, 1342 may be executed. In 1342, it is determined whether a threshold (e.g., an IP-level threshold, an FQDN-level threshold, or an owner-level threshold) should be exempted. In one embodiment, a threshold may be allowed if the corresponding IP address, FQDN, and / or owner are on the allow list and / or not on the block list.

[0285] In one embodiment, if a VCN is part of a “listening post” or “signal acquisition” honeypot, it may be permitted to exceed a threshold, in which case this VCN may be treated as control, in comparison to other VCNs for which warning and response actions are pre-programmed.

[0286] In one embodiment, observed network activity behavior may be acceptable if it matches a target pattern (e.g., a predictable walk of prefixes, known attacker profiling of hosts associated with the same namespace).

[0287] In one embodiment, conditions for allowing observed network activity may be set based on information regarding the consequences of blocking the observed network activity (for example, it may be considered more useful to use a honeypot to collect information about the observed network activity instead of blocking it). Thus, by allowing certain network activity to exceed a threshold, the allowed network activity may enable further insights to help manage subsequent network activity.

[0288] If, in 1342, it is determined that there is a valid exemption for the threshold to be exceeded, then 1344 may return to the process of checking whether another threshold (e.g., an IP-level threshold for different IP addresses, an FQDN-level threshold, an owner-level threshold) has been exceeded, and / or continuing to monitor the environment (e.g., by performing 1308).

[0289] If, in 1342, it is determined that there are no valid exemptions for the threshold to be exceeded, then in 1348, one or more actions may be performed. In one embodiment, the actions may be generating a report, sending a report, configuring a firewall, taking a system (e.g., a VCN, region, CSPI, host machine) offline, adding it to a blocklist, and / or configuring packets to be dropped.

[0290] After any action is performed, a check may be performed to see if another threshold (e.g., an IP-level threshold for different IP addresses, an FQDN-level threshold, an owner-level threshold) has been exceeded, and / or the environment may continue to be monitored (e.g., by performing 1308).

[0291] In 1326, for each unique FQDN identified in 1312, processes 1328, 1330, and 1332 may be performed.

[0292] In step 1328, the number of rDNS requests containing a unique FQDN can be determined using the extended data associated with the FQDN. The FQDN may be included in the response to the rDNS request.

[0293] In step 1330, a comparison may be made between the number of rDNS requests containing FQDNs (determined in step 1328) and the FQDN level threshold determined in step 1304. Thus, it can be determined whether the number of rDNS requests containing unique FQDNs exceeds (e.g., by a certain amount) or falls below the FQDN level threshold.

[0294] In step 1332, a determination is made as to whether the threshold for a unique FQDN at the FQDN level has been exceeded.

[0295] If the FQDN level threshold is not exceeded, processing may continue by checking whether other thresholds (e.g., FQDN level thresholds for different FQDNs, IP address level thresholds, owner level thresholds) are exceeded, and / or by continuing to monitor the environment (e.g., by running 1308).

[0296] If the FQDN level threshold is exceeded, 1342 may be executed. The process performed by 1342 has already been described above.

[0297] In 1334, for each unique owner identified in 1314, processes 1336, 1338, and 1340 may be performed.

[0298] In 1336, the number of rDNS requests containing a unique owner can be determined using extended data associated with that unique owner. The owner can be associated with the FQDN included in the response to the rDNS request and / or the IP address to which the rDNS request is directed.

[0299] At 1338, a comparison between the number of rDNS requests that include the owner (determined at 1336) and the owner-level threshold determined at 1314 may be performed. Accordingly, it may be determined whether the number of rDNS requests including the unique owner exceeds (e.g., by a certain amount) or falls below the owner-level threshold.

[0300] At 1340, a determination is made as to whether the owner-level threshold for the unique owner has been exceeded.

[0301] If the owner-level threshold has not been exceeded, processing may continue by checking whether another threshold (e.g., an owner-level threshold for a different owner, an IP address-level threshold, an FQDN-level threshold) has been exceeded, and / or continuing to monitor the environment (e.g., performing 1308), and the like.

[0302] If the owner-level threshold has been exceeded, 1342 may be executed. The processing executed at 1342 has already been described above.

[0303] The systems described herein may comprise a plurality of systems communicatively coupled to one another via one or more communication networks. Furthermore, the depicted techniques are merely examples, and are not intended to unduly limit the scope of the claimed embodiments. Many variations, alternatives, and modifications are possible to the techniques illustrated and described. The depicted systems, subsystems, and other components may be implemented in software (e.g., code, instructions, programs) executed by one or more processing units (e.g., processors, cores) of the respective systems, using hardware, or combinations thereof. The software may be stored on a non-transitory storage medium (e.g., on a memory device).

[0304] As mentioned earlier, IaaS (Infrastructure as a Service) is a specific type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the internet). In the IaaS model, a cloud computing provider can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, the IaaS provider may also provide various services that accompany those infrastructure components (examples of services include billing software, monitoring software, logging software, load balancing software, clustering software, etc.). Therefore, since these services can be policy-driven, IaaS users may be able to implement policies to drive load balancing and maintain application availability and performance.

[0305] In some cases, IaaS customers can access resources and services over a wide area network (WAN), such as the internet, and use the cloud provider's services to install the remaining elements of their application stack. For example, a user can log into the IaaS platform, create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create storage buckets for workloads and backups, and install enterprise software on the VMs. The customer can then use the provider's services to perform a variety of functions, including balancing network traffic, troubleshooting application issues, monitoring performance, and managing disaster recovery.

[0306] In most cases, the cloud computing model requires the participation of a cloud provider. A cloud provider may or may not be a third-party service specializing in providing IaaS (e.g., offering, leasing, or selling it). An entity may also choose to deploy a private cloud and become its own provider of infrastructure services.

[0307] In some cases, IaaS deployment is the process of connecting a new application or a new version of an application to a prepared application server, etc. This process may also include the process of preparing the server (e.g., installing libraries, daemons, etc.). This process is often managed by the cloud provider under the hypervisor layer (e.g., servers, storage, network hardware, and virtualization). Therefore, the customer may be responsible for handling the deployment of the OS, middleware, and / or applications (e.g., on self-service virtual machines, etc., which can be spun up on demand).

[0308] In some examples, IaaS provisioning may also refer to acquiring computers or virtual hosts for use and installing the necessary libraries or services on those computers or virtual hosts. In most cases, deployment does not include provisioning, and provisioning may need to be performed first.

[0309] In some cases, IaaS provisioning presents two distinct challenges. First, there is the initial challenge of provisioning an initial set of infrastructure before anything is executed. Second, there is the challenge of developing the existing infrastructure after everything has been provisioned (e.g., adding new services, modifying services, removing services, etc.). In some cases, these two challenges can be addressed by allowing the infrastructure configuration to be defined declaratively. In other words, the infrastructure (e.g., which components are needed and how those components interact) can be defined by one or more configuration files. In this way, the entire infrastructure topology (e.g., which resources depend on which resources and how each of those resources works together) can be described declaratively. In some cases, after the topology is defined, a workflow may be generated to create and / or manage the various components described in the configuration files.

[0310] In some examples, infrastructure can include many interconnected elements. For example, there may be one or more virtual private clouds (VPCs), also known as core networks (e.g., configurable and / or shared computing resources, possibly on-demand pools). In some examples, there may also be one or more inbound / outbound traffic group rules provisioned to define how inbound and / or outbound traffic on the network is configured, as well as one or more virtual machines (VMs). Other infrastructure elements such as load balancers and databases may also be provisioned. As more infrastructure elements are desired and / or added, the infrastructure can gradually evolve.

[0311] In some cases, continuous deployment techniques may be employed to enable the deployment of infrastructure code across various virtual computing environments. Furthermore, the techniques described can enable infrastructure management within these environments. In some examples, a service team may write code that is desirable to be deployed to one or more, but often many, different production environments (e.g., across various geographical locations, sometimes even across the entire world). However, in some examples, the infrastructure to which the code is deployed must be configured first. In some cases, provisioning can be done manually, and provisioning tools may be used to provision resources, and / or deployment tools may be used to deploy the code after the infrastructure has been provisioned.

[0312] Figure 14 is a block diagram 1600 showing an exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 1602 may be communicably coupled to a secure host tenancy 1604 which may include a virtual cloud network (VCN) 1606 and a secure host subnet 1608. In some examples, the service operator 1602 may use one or more client computing devices, which may be portable handheld devices (e.g., iPhone®, mobile phones, iPad®, computing tablets, personal digital assistants (PDAs)) or wearable devices (e.g., Google® Glass head-mounted display) with the Internet, email, short message service (SMS), Blackberry®, or other communication protocols enabled, running software such as Microsoft Windows Mobile® and / or various mobile operating systems such as iOS®, Windows Phone, Android, BlackBerry 8, Palm OS. Alternatively, a client computing device may be a general-purpose personal computer, including, for example, personal computers and / or laptop computers running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux® operating systems. A client computing device may also be a workstation computer running any of various commercially available UNIX® or UNIX-like operating systems, including, but not limited to, various GNU / Linux® operating systems such as Google Chrome OS.Alternatively or additionally, the client computing device may be any other electronic device, such as a thin client computer, an internet-enabled gaming system (e.g., a Microsoft Xbox game console with or without a Kinect® gesture input device), and / or a personal messaging device, that can communicate over a network and / or the Internet that has access to VCN1606.

[0313] VCN1606 may include a local peering gateway (LPG) 1610, which can be communicatively coupled to Secure Shell (SSH) VCN1612 via LPG1610 contained within SSH VCN1612. SSH VCN1612 may include an SSH subnet 1614, and SSH VCN1612 may be communicatively coupled to control plane VCN1616 via LPG1610 contained within control plane VCN1616. Additionally, SSH VCN1612 may be communicatively coupled to data plane VCN1618 via LPG1610. Control plane VCN1616 and data plane VCN1618 may be contained within a service tenancy 1619, which may be owned and / or operated by an IaaS provider.

[0314] The control plane VCN 1616 may include a control plane demilitarized zone (DMZ) layer 1620 that functions as a perimeter network (e.g., part of the corporate network between the corporate intranet and the external network). Servers based in the DMZ have limited responsibilities and can help keep breaches contained. Furthermore, the DMZ layer 1620 may include a control plane application layer 1624 that may include one or more load balancer (LB) subnets 1622, an application subnet 1626, and a control plane data layer 1628 that may include database (DB) subnets 1630 (e.g., a frontend DB subnet and / or a backend DB subnet). The LB subnet 1622 included in the control plane DMZ layer 1620 can be communicatively coupled to the application subnet 1626 included in the control plane application layer 1624 and to the Internet gateway 1634 which may be included in the control plane VCN 1616. The application subnet 1626 can be communicatively coupled to the DB subnet 1630 included in the control plane data layer 1628, as well as to the service gateway 1636 and the network address translation (NAT) gateway 1638. The control plane VCN 1616 may include the service gateway 1636 and the NAT gateway 1638.

[0315] The control plane VCN1616 may include a data plane mirror application layer 1640 which may include an application subnet 1626. The application subnet 1626 included in the data plane mirror application layer 1640 may include a virtual network interface controller (VNIC) 1642 which can run compute instance 1644. Compute instance 1644 can communicatively connect the application subnet 1626 of the data plane mirror application layer 1640 to an application subnet 1626 which may be included in the data plane application layer 1646.

[0316] The data plane VCN1618 may include a data plane application layer 1646, a data plane DMZ layer 1648, and a data plane data layer 1650. The data plane DMZ layer 1648 may include an LB subnet 1622 that can be communicatively coupled to the application subnet 1626 of the data plane application layer 1646 and the internet gateway 1634 of the data plane VCN1618. The application subnet 1626 may be communicatively coupled to the service gateway 1636 of the data plane VCN1618 and the NAT gateway 1638 of the data plane VCN1618. The data plane data layer 1650 may also include a DB subnet 1630 that can be communicatively coupled to the application subnet 1626 of the data plane application layer 1646.

[0317] The Internet gateways 1634 of the control plane VCN1616 and the data plane VCN1618 can be communicatively coupled to a metadata management service 1652, which can be communicatively coupled to the public internet 1654. The public internet 1654 can be communicatively coupled to the NAT gateways 1638 of the control plane VCN1616 and the data plane VCN1618. The service gateways 1636 of the control plane VCN1616 and the data plane VCN1618 can be communicatively coupled to a cloud service 1656.

[0318] In some cases, a service gateway 1636 of the control plane VCN1616 or data plane VCN1618 can make application programming interface (API) calls to a cloud service 1656 without going through the public internet 1654. API calls from the service gateway 1636 to the cloud service 1656 can be one-way, with the service gateway 1636 making the API call to the cloud service 1656, and the cloud service 1656 sending the requested data to the service gateway 1636. However, the cloud service 1656 does not need to initiate the API call to the service gateway 1636.

[0319] In some examples, secure host tenancy 1604 can be directly connected to service tenancy 1619, otherwise it may be isolated. Secure host subnet 1608 can communicate with SSH subnet 1614 via LPG 1610, which can enable bidirectional communication on otherwise isolated systems. Connecting secure host subnet 1608 to SSH subnet 1614 may give secure host subnet 1608 access to other entities within service tenancy 1619.

[0320] The control plane VCN1616 may allow users of service tenancy 1619 to configure or otherwise provision desired resources. Desired resources provisioned within the control plane VCN1616 may be deployed or otherwise used in the data plane VCN1618. In some examples, the control plane VCN1616 can be isolated from the data plane VCN1618, and the data plane mirror application layer 1640 of the control plane VCN1616 can communicate with the data plane application layer 1646 of the data plane VCN1618 via VNIC 1642, which may be included in the data plane mirror application layer 1640 and the data plane application layer 1646.

[0321] In some cases, a system user or customer can perform requests, such as create, read, update, or delete (CRUD) operations, via the public internet 1654, which can then transmit the requests to the metadata management service 1652. The metadata management service 1652 can then transmit the requests to the control plane VCN 1616 via the internet gateway 1634. The requests may be received by the LB subnet 1622, which is included in the control plane DMZ layer 1620. The LB subnet 1622 can determine that the request is valid, and in response to this determination, it can send the request to the application subnet 1626, which is included in the control plane application layer 1624. If the request is validated and requires a call to the public internet 1654, the call to the public internet 1654 may be sent to the NAT gateway 1638, which can make calls to the public internet 1654. Metadata that may be desirable to be stored by the request may be stored in the DB subnet 1630.

[0322] In some cases, the data plane mirror application layer 1640 can facilitate direct communication between the control plane VCN1616 and the data plane VCN1618. For example, it may be desirable that changes, updates, or other appropriate modifications to the configuration be applied to resources contained in the data plane VCN1618. Through VNIC1642, the control plane VCN1616 can communicate directly with the resources contained in the data plane VCN1618, thereby enabling it to perform changes, updates, or other appropriate modifications to the configuration of those resources.

[0323] In some embodiments, the control plane VCN1616 and data plane VCN1618 may be included in the service tenancy 1619. In this case, the system user or customer does not have to own or operate either the control plane VCN1616 or the data plane VCN1618. Instead, the IaaS provider may own or operate the control plane VCN1616 and data plane VCN1618, both of which may be included in the service tenancy 1619. This embodiment can enable network isolation, which can prevent the user or customer from interacting with the resources of other users or other customers. This embodiment can also enable the system user or customer to store databases privately without requiring them to rely on the public internet 1654 for storage, which may not have the desired level of threat protection.

[0324] In another embodiment, the LB subnet 1622 included in the control plane VCN 1616 may be configured to receive signals from the service gateway 1636. In this embodiment, the control plane VCN 1616 and the data plane VCN 1618 may be configured to be invoked by the IaaS provider's customer without calling the public internet 1654. The IaaS provider's customer may prefer this embodiment because the database used by the customer may be stored in a service tenancy 1619 that may be controlled by the IaaS provider and isolated from the public internet 1654.

[0325] Figure 15 is a block diagram 1700 showing another exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 1702 (e.g., service operator 1602 in Figure 16) may be communicatively coupled to a secure host tenancy 1704 (e.g., secure host tenancy 1604 in Figure 16), which may include a virtual cloud network (VCN) 1706 (e.g., VCN1606 in Figure 16) and a secure host subnet 1708 (e.g., secure host subnet 1608 in Figure 16). The VCN 1706 may include an LPG 1710 (e.g., LPG1610 in Figure 16), which may be communicatively coupled to an SSH VCN 1712 (e.g., SSH VCN1612 in Figure 16) via a local peering gateway (LPG) 1610 contained within a secure shell (SSH) VCN 1712. SSH VCN1712 may contain SSH subnet 1714 (e.g., SSH subnet 1614 in Figure 16), and SSH VCN1712 may be communicably coupled to control plane VCN1716 (e.g., control plane VCN1616 in Figure 16) via LPG1710 contained within control plane VCN1716. Control plane VCN1716 may contain service tenancy 1719 (e.g., service tenancy 1619 in Figure 16), and data plane VCN1718 (e.g., data plane VCN1618 in Figure 16) may contain customer tenancy 1721, which may be owned or operated by a user or customer of the system.

[0326] The control plane VCN1716 may include a control plane DMZ layer 1720 (e.g., control plane DMZ layer 1620 in Figure 16) which may include an LB subnet 1722 (e.g., LB subnet 1622 in Figure 16), a control plane application layer 1724 (e.g., control plane application layer 1624 in Figure 16) which may include an application subnet 1726 (e.g., application subnet 1626 in Figure 16), and a control plane data layer 1728 (e.g., control plane data layer 1628 in Figure 16) which may include a database (DB) subnet 1730 (e.g., similar to DB subnet 1630 in Figure 16). The LB subnet 1722 included in the control plane DMZ layer 1720 can be communicatively coupled to the application subnet 1726 included in the control plane application layer 1724 and to the Internet gateway 1734 (e.g., Internet gateway 1634 in Figure 16) which may be included in the control plane VCN 1716. The application subnet 1726 can be communicatively coupled to the DB subnet 1730 included in the control plane data layer 1728 and to the service gateway 1736 (e.g., service gateway 1636 in Figure 16) and the network address translation (NAT) gateway 1738 (e.g., NAT gateway 1638 in Figure 16). The control plane VCN 1716 may include the service gateway 1736 and the NAT gateway 1738.

[0327] The control plane VCN 1716 may include a data plane mirror application layer 1740 (e.g., data plane mirror application layer 1640 in Figure 16) which may include an application subnet 1726. The application subnet 1726 included in the data plane mirror application layer 1740 may include a virtual network interface controller (VNIC) 1742 (e.g., VNIC 1642) which can run a compute instance 1744 (e.g., similar to compute instance 1644 in Figure 16). The compute instance 1744 can facilitate communication between the application subnet 1726 of the data plane mirror application layer 1740 and the application subnet 1726 that may be included in the data plane application layer 1746 (e.g., data plane application layer 1646 in Figure 16) via the VNIC 1742 included in the data plane mirror application layer 1740 and the VNIC 1742 included in the data plane application layer 1746.

[0328] The Internet gateway 1734 included in the control plane VCN 1716 can be communicably coupled to a metadata management service 1752 (e.g., metadata management service 1652 in Figure 16), which can be communicably coupled to the public internet 1754 (e.g., public internet 1654 in Figure 16). The public internet 1754 can be communicably coupled to a NAT gateway 1738 included in the control plane VCN 1716. The service gateway 1736 included in the control plane VCN 1716 can be communicably coupled to a cloud service 1756 (e.g., cloud service 1656 in Figure 16).

[0329] In some examples, the data plane VCN1718 may be included in customer tenancy 1721. In this case, the IaaS provider may provide a control plane VCN1716 per customer, and the IaaS provider may configure a unique compute instance 1744 for each customer, included in service tenancy 1719. Each compute instance 1744 may enable communication between the control plane VCN1716 included in service tenancy 1719 and the data plane VCN1718 included in customer tenancy 1721. The compute instance 1744 may enable resources provisioned within the control plane VCN1716 included in service tenancy 1719 to be deployed, or otherwise used, in the data plane VCN1718 included in customer tenancy 1721.

[0330] In another example, an IaaS provider's customer may have a database that persists in customer tenancy 1721. In this example, the control plane VCN 1716 may include a data plane mirror app layer 1740 that can include an app subnet 1726. The data plane mirror app layer 1740 may reside in data plane VCN 1718, but does not have to reside in data plane VCN 1718. That is, the data plane mirror app layer 1740 may have access rights to customer tenancy 1721, but does not have to reside in data plane VCN 1718, and does not have to be owned or operated by the IaaS provider's customer. The data plane mirror app layer 1740 may be configured to make calls to data plane VCN 1718, but does not have to be configured to make calls to any entities contained in control plane VCN 1716. Customers may wish to deploy or otherwise use resources in the data plane VCN1718 that are provisioned within the control plane VCN1716, and the data plane mirror application layer 1740 can facilitate the customer's desired deployment or other use of resources.

[0331] In some embodiments, a customer of the IaaS provider can apply filters to the data plane VCN1718. In this embodiment, the customer can determine which data plane VCN1718s are accessible, and the customer can restrict access from the data plane VCN1718 to the public internet 1754. The IaaS provider does not need to be able to apply filters or otherwise control access of the data plane VCN1718 to any external network or database. Applying filters and controls to the data plane VCN1718 included in the customer tenancy 1721 by the customer can help isolate the data plane VCN1718 from other customers and from the public internet 1754.

[0332] In some embodiments, the cloud service 1756 may be invoked by the service gateway 1736 to access services that may not exist on the public internet 1754, the control plane VCN 1716, or the data plane VCN 1718. The connection between the cloud service 1756 and the control plane VCN 1716 or the data plane VCN 1718 may not be operational or continuous. The cloud service 1756 may reside on different networks owned or operated by the IaaS provider. The cloud service 1756 may be configured to receive calls from the service gateway 1736 and not to receive calls from the public internet 1754. Some cloud services 1756 may be isolated from other cloud services 1756, and the control plane VCN 1716 may be isolated from cloud services 1756 that may not be in the same region as the control plane VCN 1716. For example, the control plane VCN 1716 may be located in "Region 1", and the "deployment 16" of the cloud service may be located in Region 1 and "Region 2". If a call to deployment 16 is made by a service gateway 1736 included in control plane VCN1716 located in region 1, this call may be sent to deployment 16 in region 1. In this example, control plane VCN1716, or deployment 16 in region 1, may or may not be communicatively coupled to deployment 16 in region 2, or otherwise not communicate with deployment 16 in region 2.

[0333] Figure 16 is a block diagram 1800 showing another exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 1802 (e.g., service operator 1602 in Figure 16) may be communicatively coupled to a secure host tenancy 1804 (e.g., secure host tenancy 1604 in Figure 16) which may include a virtual cloud network (VCN) 1806 (e.g., VCN1606 in Figure 16) and a secure host subnet 1808 (e.g., secure host subnet 1608 in Figure 16). VCN 1806 may include an LPG 1810 (e.g., LPG1610 in Figure 16) which may be communicatively coupled to SSH VCN 1812 (e.g., SSH VCN1612 in Figure 16) via an LPG 1810 contained in SSH VCN 1812. SSH VCN1812 may contain SSH subnet 1814 (e.g., SSH subnet 1614 in Figure 16), and SSH VCN1812 may be communicably coupled to control plane VCN1816 (e.g., control plane VCN1616 in Figure 16) via LPG1810 contained in control plane VCN1816, and to data plane VCN1818 (e.g., data plane 1618 in Figure 16) via LPG1810 contained in data plane VCN1818. Control plane VCN1816 and data plane VCN1818 may be contained in service tenancy 1819 (e.g., service tenancy 1619 in Figure 16).

[0334] The control plane VCN1816 may include a control plane DMZ layer 1820 (e.g., control plane DMZ layer 1620 in Figure 16) which may include a load balancer (LB) subnet 1822 (e.g., LB subnet 1622 in Figure 16), a control plane application layer 1824 (e.g., control plane application layer 1624 in Figure 16) which may include an application subnet 1826 (e.g., similar to application subnet 1626 in Figure 16), and a control plane data layer 1828 (e.g., control plane data layer 1628 in Figure 16) which may include a DB subnet 1830. The LB subnet 1822 included in the control plane DMZ layer 1820 can be communicatively coupled to the application subnet 1826 included in the control plane application layer 1824, and to an internet gateway 1834 (e.g., internet gateway 1634 in Figure 16) which may be included in the control plane VCN 1816. The application subnet 1826 can be communicatively coupled to the DB subnet 1830 included in the control plane data layer 1828, and to a service gateway 1836 (e.g., service gateway in Figure 16) and a network address translation (NAT) gateway 1838 (e.g., NAT gateway 1638 in Figure 16). The control plane VCN 1816 may include the service gateway 1836 and the NAT gateway 1838.

[0335] The data plane VCN1818 may include a data plane application layer 1846 (e.g., data plane application layer 1646 in Figure 16), a data plane DMZ layer 1848 (e.g., data plane DMZ layer 1648 in Figure 16), and a data plane data layer 1850 (e.g., data plane data layer 1650 in Figure 16). The data plane DMZ layer 1848 may include a trusted application subnet 1860 and an untrusted application subnet 1862 of the data plane application layer 1846, as well as an LB subnet 1822 that can be communicatively coupled to an internet gateway 1834 included in the data plane VCN1818. The trusted application subnet 1860 may be communicatively coupled to a service gateway 1836 included in the data plane VCN1818, a NAT gateway 1838 included in the data plane VCN1818, and a DB subnet 1830 included in the data plane data layer 1850. The untrusted application subnet 1862 can be communicatively coupled to the service gateway 1836, which is included in the data plane VCN 1818, and to the DB subnet 1830, which is included in the data plane data layer 1850. The data plane data layer 1850 may include the DB subnet 1830, which can be communicatively coupled to the service gateway 1836, which is included in the data plane VCN 1818.

[0336] An untrusted application subnet 1862 may include one or more primary VNICs 1864(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 1866(1)-(N). Each tenant VM 1866(1)-(N) may be communicatively coupled to each application subnet 1867(1)-(N) that may be included in each container output VCN 1868(1)-(N) that may be included in each customer tenancy 1870(1)-(N). Each secondary VNIC 1872(1)-(N) can facilitate communication between the untrusted application subnet 1862 included in the data plane VCN 1818 and the application subnet included in the container output VCN 1868(1)-(N). Each container output VCN 1868(1)-(N) may include a NAT gateway 1838 that can be communicatively coupled to the public internet 1854 (e.g., public internet 1654 in Figure 16).

[0337] The Internet gateway 1834, which is included in the control plane VCN1816 and the data plane VCN1818, can be communicatively coupled to a metadata management service 1852 (e.g., the metadata management system 1652 in Figure 16), which can be communicatively coupled to the public internet 1854. The public internet 1854 can be communicatively coupled to a NAT gateway 1838, which is included in the control plane VCN1816 and the data plane VCN1818. The service gateway 1836, which is included in the control plane VCN1816 and the data plane VCN1818, can be communicatively coupled to a cloud service 1856.

[0338] In some embodiments, the data plane VCN1818 may be integrated with customer tenancy 1870. This integration may be useful or desirable for the IaaS provider's customer in certain cases, such as when they may want support when executing code. The customer may provide code that may be destructive, communicate with other customers' resources, or otherwise cause undesirable effects. In response to this, the IaaS provider may determine whether it should execute code provided to the IaaS provider by the customer.

[0339] In some cases, an IaaS provider's customer might request the ability to grant the IaaS provider temporary network access privileges and connect to the data plane application layer 1846. The code to perform this function may run in VMs 1866(1)-(N), and this code does not need to be configured to run elsewhere on the data plane VCN 1818. Each VM 1866(1)-(N) may be connected to one customer tenancy 1870. Each container 1871(1)-(N) contained within VMs 1866(1)-(N) may be configured to run the code. In this case, a double isolation may exist (for example, the container 1871(1)-(N) running the code, and the container 1871(1)-(N) may be contained in at least one VM 1866(1)-(N) that is in an untrusted application subnet 1862), which can help prevent incorrect or otherwise undesirable code from damaging the IaaS provider's network or the networks of different customers. Containers 1871(1)-(N) may be communicatively coupled to customer tenancy 1870 and may be configured to send or receive data to or from customer tenancy 1870. Containers 1871(1)-(N) do not need to be configured to send or receive data to or from any other entities in the data plane VCN 1818. Upon completion of code execution, the IaaS provider may terminate or otherwise discard containers 1871(1)-(N).

[0340] In some embodiments, a trusted application subnet 1860 may execute code that may be owned or operated by the IaaS provider. In this embodiment, the trusted application subnet 1860 may be communicatively coupled to a DB subnet 1830 and may be configured to perform CRUD operations within the DB subnet 1830. An untrusted application subnet 1862 may be communicatively coupled to the DB subnet 1830, but in this embodiment, the untrusted application subnet may be configured to perform read operations within the DB subnet 1830. Containers 1871(1)~(N) that may be contained within each customer's VM 1866(1)~(N) and can execute code from the customer do not need to be communicatively coupled to the DB subnet 1830.

[0341] In other embodiments, the control plane VCN1816 and the data plane VCN1818 do not need to be directly and communicatively coupled. In this embodiment, direct communication between the control plane VCN1816 and the data plane VCN1818 is not required. However, communication can occur indirectly by at least one method. The LPG1810 may be established by the IaaS provider and can facilitate communication between the control plane VCN1816 and the data plane VCN1818. In another example, the control plane VCN1816 or the data plane VCN1818 can make a call to the cloud service 1856 via the service gateway 1836. For example, a call from the control plane VCN1816 to the cloud service 1856 may include a request for a service that can communicate with the data plane VCN1818.

[0342] Figure 17 is a block diagram 1900 showing another exemplary pattern of an IaaS architecture according to at least one embodiment. A service operator 1902 (e.g., service operator 1602 in Figure 16) may be communicatively coupled to a secure host tenancy 1904 (e.g., secure host tenancy 1604 in Figure 16) which may include a virtual cloud network (VCN) 1906 (e.g., VCN1606 in Figure 16) and a secure host subnet 1908 (e.g., secure host subnet 1608 in Figure 16). VCN1906 may include an LPG1910 (e.g., LPG1610 in Figure 16) which may be communicatively coupled to SSH VCN1912 (e.g., SSH VCN1612 in Figure 16) via an LPG1910 contained in SSH VCN1912. SSH VCN1912 may contain SSH subnet 1914 (e.g., SSH subnet 1614 in Figure 16), and SSH VCN1912 may be communicably coupled to control plane VCN1916 (e.g., control plane VCN1616 in Figure 16) via LPG1910 contained in control plane VCN1916, and to data plane VCN1918 (e.g., data plane 1618 in Figure 16) via LPG1910 contained in data plane VCN1918. Control plane VCN1916 and data plane VCN1918 may be contained in service tenancy 1919 (e.g., service tenancy 1619 in Figure 16).

[0343] The control plane VCN1916 may include a control plane DMZ layer 1920 (e.g., control plane DMZ layer 1620 in Figure 16) which may include an LB subnet 1922 (e.g., LB subnet 1622 in Figure 16), a control plane application layer 1924 (e.g., control plane application layer 1624 in Figure 16) which may include an application subnet 1926 (e.g., application subnet 1626 in Figure 16), and a control plane data layer 1928 (e.g., control plane data layer 1628 in Figure 16) which may include a DB subnet 1930 (e.g., DB subnet 1830 in Figure 16). The LB subnet 1922 included in the control plane DMZ layer 1920 can be communicatively coupled to the application subnet 1926 included in the control plane application layer 1924, and to an internet gateway 1934 (e.g., internet gateway 1634 in Figure 16) which may be included in the control plane VCN 1916. The application subnet 1926 can be communicatively coupled to the DB subnet 1930 included in the control plane data layer 1928, and to a service gateway 1936 (e.g., service gateway in Figure 16) and a network address translation (NAT) gateway 1938 (e.g., NAT gateway 1638 in Figure 16). The control plane VCN 1916 may include the service gateway 1936 and the NAT gateway 1938.

[0344] The data plane VCN1918 may include a data plane application layer 1946 (e.g., data plane application layer 1646 in Figure 16), a data plane DMZ layer 1948 (e.g., data plane DMZ layer 1648 in Figure 16), and a data plane data layer 1950 (e.g., data plane data layer 1650 in Figure 16). The data plane DMZ layer 1948 may include trusted application subnets 1960 (e.g., trusted application subnet 1860 in Figure 16) and untrusted application subnets 1962 (e.g., untrusted application subnet 1862 in Figure 16) of the data plane application layer 1946, as well as an LB subnet 1922 that can be communicatively coupled to the internet gateway 1934 included in the data plane VCN1918. A trusted application subnet 1960 may be communicatively coupled to a service gateway 1936 included in data plane VCN 1918, a NAT gateway 1938 included in data plane VCN 1918, and a DB subnet 1930 included in data plane data layer 1950. An untrusted application subnet 1962 may be communicatively coupled to a service gateway 1936 included in data plane VCN 1918, and a DB subnet 1930 included in data plane data layer 1950. The data plane data layer 1950 may include a DB subnet 1930 that can be communicatively coupled to a service gateway 1936 included in data plane VCN 1918.

[0345] An untrusted application subnet 1962 may include primary VNICs 1964(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 1966(1)-(N) residing within the untrusted application subnet 1962. Each tenant VM 1966(1)-(N) can execute code within its respective container 1967(1)-(N) and may be communicatively coupled to an application subnet 1926 that may be contained in a dataplane application layer 1946 that may be contained in a container output VCN 1968. Each secondary VNIC 1972(1)-(N) can facilitate communication between the untrusted application subnet 1962 contained in a dataplane VCN 1918 and the application subnet contained in a container output VCN 1968. The container output VCN may include a NAT gateway 1938 that can be communicatively coupled to the public internet 1954 (e.g., public internet 1654 in Figure 16).

[0346] The Internet gateway 1934, included in the control plane VCN1916 and the data plane VCN1918, can be communicatively coupled to a metadata management service 1952 (e.g., the metadata management system 1652 in Figure 16), which can be communicatively coupled to the public internet 1954. The public internet 1954 can be communicatively coupled to a NAT gateway 1938, included in the control plane VCN1916 and the data plane VCN1918. The service gateway 1936, included in the control plane VCN1916 and the data plane VCN1918, can be communicatively coupled to a cloud service 1956.

[0347] In some examples, the pattern shown by the architecture in block diagram 1900 of Figure 17 may be considered an exception to the pattern shown by the architecture in block diagram 1800 of Figure 16, which may be desirable for the IaaS provider's customers when the IaaS provider cannot communicate directly with the customers (e.g., disconnected regions). Each container 1967(1)-(N) contained within VM1966(1)-(N) for each customer may be accessed in real time by the customer. Each container 1967(1)-(N) may be configured to make calls to each secondary VNIC 1972(1)-(N) contained in the application subnet 1926 of the data plane application layer 1946, which may be contained within the container output VCN 1968. The secondary VNICs 1972(1)-(N) may send calls to the NAT gateway 1938, which may send calls to the public internet 1954. In this example, containers 1967(1)-(N), which can be accessed in real time by customers, can be isolated from the control plane VCN1916 and from other entities included in the data plane VCN1918. Containers 1967(1)-(N) can also be isolated from other customer resources.

[0348] In another example, a customer can use containers 1967(1)-(N) to invoke cloud service 1956. In this example, the customer may execute code within containers 1967(1)-(N) that requests a service from cloud service 1956. Containers 1967(1)-(N) can send this request to secondary VNICs 1972(1)-(N), which can send this request to the NAT gateway, which can send this request to the public internet 1954. The public internet 1954 can send this request to the LB subnet 1922, which is included in the control plane VCN 1916, via the internet gateway 1934. In response to determining that this request is valid, the LB subnet can send this request to the application subnet 1926, which can send this request to cloud service 1956 via the service gateway 1936.

[0349] It should be understood that the IaaS architectures 1600, 1700, 1800, and 1900 shown in the figures may include components other than those shown. Furthermore, the embodiments shown in the figures are merely examples of cloud infrastructure systems that may incorporate certain embodiments of this disclosure. In some other embodiments, the IaaS system may include more or fewer components than those shown in the figures, may combine two or more components, or may have different configurations or arrangements of components.

[0350] In some embodiments, the IaaS system described herein may include the provision of a set of applications, middleware, and database services delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such an IaaS system is the Oracle Cloud Infrastructure (OCI) provided by the Assignee.

[0351] Figure 18 shows an exemplary computer system 2000 in which various embodiments may be implemented. System 2000 may be used to implement any of the computer systems described above. As shown in the figure, computer system 2000 includes a processing unit 2004 that communicates with several peripheral subsystems via a bus subsystem 2002. These peripheral subsystems may include a processing acceleration unit 2006, an I / O subsystem 2008, a storage subsystem 2018, and a communication subsystem 2024. The storage subsystem 2018 includes a tangible computer-readable storage medium 2022 and system memory 2010.

[0352] The bus subsystem 2002 provides a mechanism for various components and subsystems of the computer system 2000 to communicate with each other as intended. Although the bus subsystem 2002 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. The bus subsystem 2002 may be one of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of the various bus architectures. For example, such architectures may include the ISA (Industry Standard Architecture) bus, the MCA (Micro Channel Architecture) bus, the EISA (Enhanced ISA) bus, the VESA (Video Electronics Standards Association) local bus, and the PCI (Peripheral Component InterConnect) bus, which can be implemented as a mezzanine bus manufactured according to the IEEE P1386.1 standard.

[0353] A processing unit 2004, which can be implemented as one or more integrated circuits (e.g., conventional microprocessors or microcontrollers), controls the operation of the computer system 2000. One or more processors may be included in the processing unit 2004. These processors may include single-core processors or multi-core processors. In one embodiment, the processing unit 2004 may be implemented as one or more independent processing units 2032 and / or 2034, each processing unit containing a single-core processor or a multi-core processor. In another embodiment, the processing unit 2004 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors onto a single chip.

[0354] In various embodiments, the processing unit 2004 can execute various programs depending on the program code and can maintain multiple programs or processes running simultaneously. At any given time, some or all of the program code to be executed may reside in the processor 2004 and / or the storage subsystem 2018. With appropriate programming, the processor 2004 can provide the various functions described above. The computer system 2000 may further include a processing acceleration unit 2006 which may include a digital signal processor (DSP), a dedicated processor, and / or similar.

[0355] The I / O subsystem 2008 may include user interface input devices and user interface output devices. User interface input devices may include pointing devices such as keyboards, mice or trackballs, touchpads or touchscreens integrated into displays, scroll wheels, click wheels, dials, buttons, switches, keypads, voice input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may also include motion detection devices and / or gesture recognition devices, such as the Microsoft Kinect® motion sensor, which enables users to control and interact with input devices such as Microsoft Xbox® 360 game controllers through a natural user interface using gestures and spoken commands. User interface input devices may also include eye gesture recognition devices, such as the Google® Glass blink detector, which detects the user's eye activity (e.g., blinking when taking a picture and / or selecting a menu) and translates eye gestures as input to an input device (e.g., Google® Glass). Furthermore, the user interface input device may include a voice recognition detection device that allows the user to interact with a voice recognition system (e.g., Siri® Navigator) via voice commands.

[0356] User interface input devices may include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, gamepads, and graphic tablets, as well as audio / visual devices such as speakers, digital cameras, digital video cameras, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. Furthermore, user interface input devices may include medical imaging input devices such as computed tomography, magnetic resonance imaging, positional emission tomography, and medical ultrasound imaging devices. User interface input devices may also include audio input devices such as MIDI keyboards and digital musical instruments.

[0357] User interface output devices may include non-visual displays such as display subsystems, indicator lights, or audio output devices. Display subsystems may include flat panel devices such as flat panel devices using cathode ray tubes (CRTs), liquid crystal displays (LCDs), or plasma displays, projection devices, touchscreens, etc. Generally, the use of the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from the computer system 2000 to a user or another computer. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text information, graphics information, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, audio output devices, and modems.

[0358] The computer system 2000 may include a storage subsystem 2018 that provides a tangible, non-temporary, computer-readable storage medium for storing software and data structures that provide the functionality of the embodiments described herein. The software may include programs, code modules, instructions, scripts, etc., and, when executed by one or more cores or processors of the processing unit 2004, provides the aforementioned functionality. The storage subsystem 2018 may also provide a repository for storing data used in accordance with this disclosure.

[0359] As shown in the example in Figure 18, the storage subsystem 2018 may include various components, including system memory 2010, computer-readable storage media 2022, and computer-readable storage media reader 2020. System memory 2010 may store program instructions that can be read and executed by processing unit 2004. System memory 2010 may also store data used during the execution of instructions and / or data generated during the execution of program instructions. Various different types of programs may be loaded into system memory 2010, including, but not limited to, client applications, web browsers, middle-tier applications, relational database management systems (RDBMS), virtual machines, containers, etc.

[0360] System memory 2010 may also store operating systems 2016. Examples of operating systems 2016 include various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux® operating systems, various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux® operating systems, Google Chrome® OS, etc.), and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® OS, and Palm® OS. In some implementations where computer system 2000 runs one or more virtual machines, the virtual machines, along with guest operating systems (GOS), may be loaded into system memory 2010 and run by one or more processors or cores of processing unit 2004.

[0361] System memory 2010 may be provided in different configurations depending on the type of computer system 2000. For example, system memory 2010 may be volatile memory (random access memory (RAM) etc.) and / or non-volatile memory (read-only memory (ROM) flash memory etc.). Various types of RAM configurations may be provided, including static random access memory (SRAM), dynamic random access memory (DRAM), etc. In some implementations, system memory 2010 may include a basic input / output system (BIOS) that contains basic routines that help transfer information between elements within the computer system 2000 during startup, etc.

[0362] Computer-readable storage media 2022 may represent, in addition to storage media for temporarily and / or more permanently containing and storing computer-readable information for use by computer system 2000, including instructions executable by processing unit 2004 of computer system 2000, remote storage devices, local storage devices, fixed storage devices, and / or removable storage devices.

[0363] Computer-readable storage medium 2022 may include any suitable medium known or used in the art, including, but not limited to, storage and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media, implemented in any way or technique for storing and / or transmitting information. Computer-readable storage medium 2022 may include tangible computer-readable storage media such as RAM, ROM, electronically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD), or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage, or other magnetic storage devices, or other tangible computer-readable media.

[0364] For example, Computer-Readable Storage Media 2022 may include hard disk drives that read from or write to non-removable non-volatile magnetic media, magnetic disk drives that read from or write to removable non-volatile magnetic disks, and optical disk drives that read from or write to removable non-volatile optical disks such as CD-ROMs, DVDs, and Blu-ray® discs, or other optical media. Computer-Readable Storage Media 2022 may also include, but is not limited to, Zip® drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVD discs, and digital videotapes. Computer-readable storage media 2022 may include solid-state drives (SSDs) based on non-volatile memory such as flash memory-based SSDs, enterprise flash drives, and semiconductor ROM; SSDs based on volatile memory such as semiconductor RAM, dynamic RAM, static RAM, DRAM-based SSDs, and magneto-resistive RAM (MRAM) SSDs; and hybrid SSDs using a combination of DRAM and flash memory-based SSDs. Disk drives and associated computer-readable media may provide non-volatile storage for computer-readable instructions, data structures, program modules, and other data of the computer system 2022.

[0365] Machine-readable instructions executable by one or more processors or cores of the processing unit 2004 may be stored in a non-temporary computer-readable storage medium. The non-temporary computer-readable storage medium may include physically tangible memory or storage devices, including volatile memory storage devices and / or non-volatile storage devices. Examples of non-temporary computer-readable storage media include magnetic storage media (e.g., disks or tapes), optical storage media (e.g., DVDs, CDs), various types of RAM, ROM, or flash memory, hard drives, floppy disk drives, removable memory drives (e.g., USB drives), or other types of storage devices.

[0366] The communications subsystem 2024 provides interfaces to other computer systems and networks. It functions as an interface for computer system 2000 to receive data from other systems and to transmit data to other systems. For example, the communications subsystem 2024 may enable computer system 2000 to connect to one or more devices via the Internet. In some embodiments, the communications subsystem 2024 may include radio frequency (RF) transceiver components for accessing wireless voice and / or data networks (e.g., advanced data network technologies such as cellular technology, 3G, 4G, or EDGE (enhanced data rates for global evolution)), Wi-Fi (using IEEE 802.11 group standards, or other mobile communication technologies, or any combination thereof), global positioning system (GPS) receiver components, and / or other components. In some embodiments, the communications subsystem 2024 may provide wired network connectivity (e.g., Ethernet) in addition to, or instead of, wireless interfaces.

[0367] In some embodiments, the communication subsystem 2024 may also receive input communications in the form of structured and / or unstructured data feeds 2026, event streams 2028, event updates 2030, etc., on behalf of one or more users who may be using the computer system 2000.

[0368] For example, the communications subsystem 2024 may be configured to receive data feeds 2026 in real time from users of social networks and / or other communications services such as Twitter® feeds and Facebook® updates, web feeds such as Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party sources.

[0369] Furthermore, the communication subsystem 2024 may be configured to receive data in the form of a continuous data stream, which may include an event stream 2028 and / or event update 2030 of real-time events that are inherently continuous or boundaryless, without an explicit end. Examples of applications that generate continuous data include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, and automotive traffic monitoring.

[0370] The communication subsystem 2024 may also be configured to output structured and / or unstructured data feeds 2026, event streams 2028, event updates 2030, etc., to one or more databases that can communicate with one or more streaming data source computers coupled to the computer system 2000.

[0371] Computer System 2000 can be one of a variety of types, including handheld portable devices (e.g., iPhone® mobile phones, iPad® computing tablets, PDAs), wearable devices (e.g., Google® Glass head-mounted displays), PCs, workstations, mainframes, automated ticket machines, server racks, or any other data processing systems.

[0372] Due to the constantly changing nature of computers and networks, the description of the computer system 2000 shown in the figure is intended to be merely an example. Many other configurations are possible, including more or fewer components than the system shown in the figure. For example, customized hardware may be used, and / or certain elements may be implemented in hardware, firmware, software (including applets), or a combination thereof. Furthermore, connections to other computing devices, such as network input / output devices, may be employed. Based on the disclosures and teachings provided herein, those skilled in the art will understand other ways and / or methods for carrying out various embodiments.

[0373] While specific embodiments have been described, various modifications, changes, alternative structures, and equivalents are also included within the scope of this disclosure. The embodiments are not limited to operation within a particular data processing environment, but can freely operate within multiple data processing environments. Furthermore, while the embodiments have been described using a specific set of transactions and steps, it should be apparent to those skilled in the art that the scope of this disclosure is not limited to the described set of transactions and steps. The various features and aspects of the embodiments described above may be used individually or in combination.

[0374] Furthermore, while embodiments have been described using specific combinations of hardware and software, it should be recognized that other combinations of hardware and software are also within the scope of this disclosure. Embodiments may be implemented using hardware alone, software alone, or a combination thereof. Various processes described herein may be performed on different processors, either on the same processor or in any combination thereof. Thus, where a component or service is described as being configured to perform a certain operation, such configuration may be realized, for example, by designing electronic circuitry to perform this operation, by programming programmable electronic circuitry (such as a microprocessor) to perform this operation, or by any combination thereof. Processes may communicate using a variety of techniques, including but not limited to prior art for inter-process communication, different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0375] Therefore, this specification and the drawings should be considered illustrative, not limiting. However, it is clear that additions, reductions, deletions, and other modifications and changes may be made to this specification and the drawings without departing from the broader idea and scope set forth in the claims. Thus, while certain embodiments of the disclosure have been described, they are not intended to be limiting. Various modifications and equivalents are within the scope of the appended claims.

[0376] The terms “a,” “an,” and “the” and similar reference subjects used in the context describing the disclosed embodiments (in particular in the context of the appended claims) should be construed to apply to both singular and plural nouns unless otherwise specifically indicated herein or unless it is clearly inconsistent with the context. The terms “equipped,” “having,” “including,” and “containing” should be construed to be unrestricted terms (i.e., “including, but not limited to,” “including,” but not limited to,” unless otherwise specifically noted. The term “connected” should be construed to mean partially or completely contained, connected, or joined together, even if there is something intervening. The enumeration of ranges of values ​​herein is intended simply as a way to refer individually to each separate value contained within the range unless otherwise specifically indicated herein, and each separate value is incorporated herein as if it were individually enumerated herein. All methods described herein may be performed in any appropriate order unless otherwise specifically indicated herein or unless it is clearly inconsistent with the context. Any use of any examples or illustrative language (e.g., "etc.") provided herein is intended solely to better illustrate the embodiments and, unless otherwise claimed, does not limit the scope of this disclosure. The language herein should not be construed as indicating that any unclaimed element is essential to the practice of this disclosure.

[0377] Disjunctive language, such as the phrase "at least one of X, Y, or Z," is generally intended to be understood in context to indicate that an item, condition, etc., can be any one of X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z), unless otherwise explicitly stated. Therefore, such disjunctive language is not generally intended, and should not, mean that a particular embodiment requires the presence of at least one of X, at least one of Y, or at least one of Z, respectively.

[0378] This specification describes preferred embodiments of the Disclosure, including known best modes for carrying out the Disclosure. As a person skilled in the art reads the foregoing description, variations of such preferred embodiments may become apparent. A person skilled in the art should be able to adopt such variations as needed, and the Disclosure may be practiced in ways other than those described in detail herein. Accordingly, this Disclosure includes all modifications and equivalents of the subject matter enumerated in the claims appended herein, as permitted by applicable law. Furthermore, any combination of the aforementioned elements in all possible variations of the embodiments is encompassed by this Disclosure unless specifically indicated herein.

[0379] All references cited herein, including published documents, patent applications, and patents, are individually and in detail incorporated by reference, and are incorporated herein by reference to the same extent as if they were presented as a whole. While the embodiments of this disclosure have been described in the foregoing specification with reference to specific embodiments thereof, those skilled in the art will recognize that this disclosure is not limited thereto. The various features and embodiments of the foregoing disclosure may be used individually or in combination. Furthermore, embodiments may be used in any number of environments and applications beyond those described herein without departing from the broader idea and scope of this specification. Accordingly, this specification and the drawings should be considered illustrative rather than restrictive.

Claims

1. A method by which a computer performs an action. The cloud defense system includes monitoring reverse DNS traffic related to a monitored environment, wherein the reverse DNS traffic includes one or more sets of reverse DNS resolver requests originating from the monitored environment, and one or more sets of responses generated by one or more DNS resolvers in response to the one or more sets of reverse DNS resolver requests, and the method performed by the computer further includes: The cloud defense system monitors one or more responses to one or more sets of reverse DNS resolver requests, The cloud defense system collects and stores raw data based on the monitoring of the reverse DNS traffic. The aforementioned cloud defense system generates enhanced data by expanding the raw data, The cloud defense system uses the augmented data to identify irregular network activity related to the monitored environment, A method performed by a computer, comprising the cloud defense system outputting a signal indicating the irregular network activity.

2. The method performed by a computer according to claim 1, wherein identifying the irregular network activity includes identifying the portion of the monitored environment in which the irregular network activity is occurring, the portion of the monitored environment comprising one or more components of the monitored environment.

3. The method performed by a computer according to claim 2, wherein the one or more components of the monitored environment include at least one of a virtual cloud network (VCN) in the monitored environment, a region in the monitored environment, a set of one or more VCNs associated with a customer of a cloud service provider, a data center in the monitored environment, a virtual machine, and a host machine.

4. Identifying the irregular network activity is a method performed by a computer according to any of the preceding claims, comprising identifying the source of the irregular network activity.

5. The computer method according to claim 4, wherein the source is at least one of (i) a part of the monitored environment or (ii) a component outside the monitored environment.

6. A method performed by a computer according to claim 4 or 5, wherein identifying the source includes identifying at least one of the following: a first IP address associated with the source that caused at least one reverse DNS request from the set of one or more reverse DNS resolver requests; a first fully qualified domain name (FQDN) associated with the first IP address; or an owner associated with the first FQDN.

7. A computer method according to any of the preceding claims, further comprising initiating one or more sets of actions in response to the cloud defense system outputting the signal indicating the irregular network activity.

8. A method performed by a computer according to claim 7, wherein the set of one or more actions performs at least one of the following: (i) modifying a set of rules associated with a component of the cloud defense system; (ii) isolating the system within the cloud server provider infrastructure (CSPI); and (iii) causing an alert to be generated.

9. The method performed by the computer according to claim 8, wherein the warning is a report and the warning is sent to a user of the cloud defense system or a customer of the CSPI.

10. Identifying the aforementioned irregular network activity is This includes generating a first baseline using pre-enhanced data, wherein the pre-enhanced data is generated before generating the first baseline. Determining the deviation from the first baseline, A method performed by a computer according to any of the preceding claims, further comprising identifying the deviation as the irregular network activity.

11. The first baseline identifies the portion of the monitored environment and the first threshold associated with the portion of the monitored environment. A method performed by a computer according to claim 10, wherein determining the deviation includes determining, based on the extended data, that the first threshold associated with the portion has been exceeded.

12. The method performed by a computer according to claim 11, wherein the first baseline represents the number of rDNS requests in the set of one or more rDNS resolver requests transmitted by the portion of the monitored environment.

13. The method performed by a computer according to claim 11, wherein the first baseline represents the number of rDNS requests in the set of one or more rDNS resolver requests sent by the portion of the monitored environment to resolve one or more sets of IP addresses.

14. The computer-operated method according to claim 11, wherein the first baseline is different from a second baseline which identifies a second portion of the monitored environment using a second threshold different from the first threshold.

15. The set of one or more reverse DNS resolver requests is generated by one or more VCNs, one or more regions, or one or more virtual machines, as performed by the computer according to any of the preceding claims.

16. A method performed by a computer according to any of the preceding claims, wherein the raw data and an external data source are used when generating the extended data.

17. It is a cloud-based defense system, A traffic monitoring system is provided, the traffic monitoring system monitors reverse DNS traffic related to a monitored environment and obtains raw data, the reverse DNS traffic includes one or more sets of reverse DNS resolver requests originating from the monitored environment, and one or more sets of responses generated by one or more DNS resolvers in response to the one or more sets of reverse DNS resolver requests. The cloud defense system includes a data augmentation system that generates augmented data using the raw data, A cloud defense system further comprising: a data analysis system that uses the aforementioned enhanced data to generate one or more warnings or one or more reports, or to identify one or more patterns within the aforementioned enhanced data.

18. The cloud defense system according to claim 17, wherein the data analysis system identifies the portion of the monitored environment where irregular network activity is occurring.

19. The cloud defense system according to claim 17 or 18, wherein the traffic monitoring system monitors reverse DNS resolver requests generated by one or more VCNs, one or more regions, or one or more virtual machines.

20. A non-temporary computer-readable medium for storing a set of instructions, wherein the set of instructions, when executed by one or more processors, causes a process to be performed, and the process is: The cloud defense system includes monitoring reverse DNS traffic related to a monitored environment, wherein the reverse DNS traffic includes one or more sets of reverse DNS resolver requests originating from the monitored environment, and one or more sets of responses generated by one or more DNS resolvers in response to the one or more sets of reverse DNS resolver requests. The cloud defense system monitors one or more responses to one or more sets of reverse DNS resolver requests, The aforementioned cloud defense system collects and stores raw data based on the monitoring of the reverse DNS traffic, The aforementioned cloud defense system expands the raw data to generate augmented data, The cloud defense system uses the augmented data to identify irregular network activity related to the monitored environment, A non-temporary computer-readable medium further comprising the cloud defense system outputting a signal indicating the irregular network activity.

Images