A technique for automatically configuring minimum cloud service access rights for container applications

Abstract

The computer system may receive one or more requests to access one or more cloud services and may store the one or more requests in a request log. The computer system may receive one or more access rules applicable to cloud service access rights. The computer system may aggregate the one or more requests in the request log to determine access requirements for a container, the container configured to store one or more applications. The computer system may generate and store a container access policy that defines access to the container and the one or more cloud services, the container access policy based at least in part on the aggregated one or more requests and the one or more access rules. The computer system may send the container access policy to a request forwarder of a compute instance in the production environment.

Description

[Technical field]

[0001] CROSS-REFERENCE TO PRIOR APPLICATIONS This application is a PCT application of U.S. Non-provisional Application No. 17 / 177,159, filed on February 16, 2021, and entitled “TECHNIQUES FOR AUTOMATICALLY CONFIGURING MINIMAL CLOUD SERVICE ACCESS RIGHTS FOR CONTAINER APPLICATIONS,” and claims the benefit of and priority to such non-provisional application under 35 U.S.C. § 119(e), the entire contents of which are incorporated by reference for all purposes. [Background technology]

[0002] background Infrastructure and platform-as-a-service cloud providers support federated authentication. In particular, customer workloads hosted on the cloud provider's infrastructure can be authenticated using the identity of the compute instance hosted on it. This model can enable customers to build secure workloads. Without it, customers would need to bootstrap instances or applications running on them with credentials every time they are created and ensure that such credentials are recognized by the cloud service.

[0003] Customer access rules can limit access by compute instances or nodes to specific services or customer data to prevent all containers from having the same level of access to the cloud infrastructure. Existing systems for establishing cloud service connectivity often use overly permissive rules where all containers have the same level of access to the cloud infrastructure. Summary of the Invention

[0004] Quick Overview [Problem to be solved by the invention]

[0005] Certain embodiments of the present disclosure may provide a method, system, and computer-readable storage medium for managing access to cloud-based services. The present disclosure describes systems and techniques for mediating requests between containers and cloud services to provide an appropriate level of access control. The disclosed techniques combine cloud orchestrator internal authentication to identify the container caller with cloud service authentication to authenticate calls made to the cloud service. In some examples, individual containers running on a system may not directly access metadata services or instance credentials. Instead, containers may send requests through a request forwarder component. That component establishes the container identity and verifies that a particular container has access to communicate with a target service (e.g., a cloud service). The request forwarder component uses the instance credential to authenticate calls to the target service. Compute instances may be configured to prevent containers from accessing metadata services. This technique effectively prevents processes in a container from using the compute instance credential. Compute instances are examples of virtual processors, compute nodes in a cloud, and even bare metal processors (e.g., physical hardware computers).

[0006] When a container is initialized, it may be provided with credentials by a container orchestrator. The container orchestrator may use the credentials to identify the container in addition to replacing a node, replacing a container to a node, etc. This process may take a variety of forms. In some examples, this process results in the credentials being stored in the container's file system. According to one aspect of the present disclosure, a process running within a container intends to make a call to a cloud service. The process may send a request for the cloud service to a request forwarder. The request may include container credentials. The request forwarder may receive the request and determine the identity of the container by sending the container credentials to the container orchestrator. The request forwarder may verify that the container is authorized to access the target cloud service by referencing one or more policies stored in the system. The request forwarder may retrieve instance credentials from a metadata service. The request forwarder may send the request including the compute instance credentials to the target cloud service. The cloud service may validate the request against one or more stored cloud policies to verify that the instance running within the container is authorized to access the requested cloud service.

[0007] Specifically, a container can run one or more applications that may require access to cloud resources. For example, the resources may be used to create virtual machines, or to access sequences or data objects, or to manage keys in a key management system, or to store data in a database. The cloud infrastructure system may include its own unified access management mechanism. In the access management system, access can be granted to a principal. While it is technically possible to simply create a principal per container, that technique would require including granting access to each container. This technique is difficult to replicate at scale, since it would require provisioning credentials and partitioning those containers. This problem can only be solved by using compute nodes. Compute nodes have their own first-class identity, and access can be granted to those nodes, allowing those machines to perform these kinds of actions as principals.

[0008] The disclosed technology allows for container isolation, assuming there are two pieces of data that need to be stored separately. For example, one container can store the first piece of data and a second container can store the second piece of data. Ideally, the containers are not hosted on the same machine. This way, if there is a breach or vulnerability in one container or container isolation, the vulnerability does not provide access to other data in other containers. [Means for solving the problem]

[0009] In some aspects, a method includes receiving (e.g., from a source) one or more requests to access one or more cloud services, storing the one or more requests in a request log, receiving (e.g., from different sources or the same source) one or more access rules applicable to cloud service access rights, and aggregating the one or more requests from the request log to determine access requirements for a container, the container being configured to store one or more applications, the method further includes generating a container access policy that defines access of the container and the one or more cloud services, the container access policy being based at least in part on the aggregated one or more requests and the one or more access rules, the method further includes storing the container access policy in a memory and sending the container access policy to a request forwarder of a compute instance in a production environment, the request forwarder accessing the container access policy to grant the container access to the one or more cloud services.

[0010] In some aspects, a node access policy is generated, the node access policy specifying an access policy for granting combined access of one or more containers on a node to a group of compute instances, and further, the container access policy is stored in the memory.

[0011] In some aspects, the method includes granting access permissions equal to the combined access of the one or more containers assigned to the node.

[0012] In some aspects, the method includes dividing the compute instances having a plurality of compute instances into a group of nodes, each of the nodes having separate access, and the method further includes assigning one or more containers to nodes having sufficient access based at least in part on the node access policy.

[0013] In some aspects, node access is predetermined and container access within each node is dynamically set.

[0014] In some aspects, the method includes testing access requirements of a cloud system, detecting that a particular application failed to access the one or more cloud services based at least in part on entries in the request log, and modifying permissions of a compute instance to correct the failure.

[0015] In some aspects, the method includes setting the request forwarder for each container type in the production environment to an allow mode, the allow mode granting access to the one or more cloud services for the one or more applications stored in the container, the method further includes receiving the one or more requests from the request forwarder and, in response to a number of the one or more requests exceeding a threshold requirement, switching the request forwarder to a restrictive mode, the restrictive mode granting access to the one or more cloud services for the one or more applications based in part on the container access policy, the method further includes switching the request forwarder to a restrictive mode, in response to a number of the one or more requests exceeding a threshold requirement, the restrictive mode granting access to the one or more cloud services for the one or more applications based in part on the container access policy.

[0016] In some aspects, a non-transitory computer-readable medium storing a set of instructions for configuring cloud service access rights for a container application includes one or more instructions that, when executed by one or more processors of a computer system, cause the computer system to receive one or more requests (e.g., from a source) to access one or more cloud services, store the one or more requests in a request log, receive one or more access rules (e.g., from different sources or the same source) applicable to cloud service access rights, and aggregate the one or more requests in the request log to determine access requirements for a container, where the container receives one or more access rules. and configured to store the application on a production environment, the computer system further causing the computer system to generate a container access policy defining access of a container and the one or more cloud services, the container access policy being based at least in part on the aggregated one or more requests and the one or more access rules, the computer system further causing the computer system to store the container access policy in a memory and send the container access policy to a request forwarder of a compute instance in a production environment, the request forwarder accessing the container access policy to grant access to the container to the one or more cloud services.

[0017] In some aspects, the one or more instructions further cause the computer system to generate a node access policy, the node access policy specifying an access policy for granting combined access of one or more containers on a node to a group of compute instances, and further cause the computer system to store the container access policy in the memory.

[0018] In some aspects, the one or more instructions further cause the computer system to grant access permissions equal to the combined access of the one or more containers assigned to the node.

[0019] In some aspects, the one or more instructions further cause the computer system to divide the compute instances, having a plurality of compute instances, into a group of nodes, each of the nodes having separate access, and further cause the computer system to assign one or more containers to nodes having sufficient access based at least in part on the node access policy.

[0020] In some aspects, node access is predetermined and container access within each node is dynamically set.

[0021] In some aspects, the one or more instructions further cause the computer system to test access requirements of a cloud system, detect that a particular application failed to access the one or more cloud services based at least in part on entries in the request log, and modify permissions of a computing instance to correct the failure.

[0022] In some aspects, the one or more instructions further cause the computer system to set the request forwarder for each container type in the production environment to an allow mode, where the allow mode grants access to the one or more cloud services for the one or more applications stored in the container, and further cause the computer system to receive the one or more requests from the request forwarder and, in response to a number of the one or more requests exceeding a threshold requirement, switch the request forwarder to a restrictive mode, where the restrictive mode grants access to the one or more cloud services for the one or more applications based in part on the container access policy.

[0023] In some aspects, a computer system includes one or more memories and one or more processors communicatively coupled to the one or more memories and configured to perform operations for setting cloud service access rights for a container application, the operations including receiving (e.g., from a source) one or more requests to access one or more cloud services, storing the one or more requests in a request log, receiving (e.g., from different sources or a same source) one or more access rules applicable to cloud service access rights, and aggregating the one or more requests in the request log to determine access requirements for a container, the container being configured to store one or more applications, the operations further including generating a container access policy that defines access for the container and the one or more cloud services, the container access policy being based at least in part on the aggregated one or more requests and the one or more access rules, the operations further including storing the container access policy in memory and sending the container access policy to a request forwarder of a compute instance in a production environment, the request forwarder accessing the container access policy to grant the container access rights to the one or more cloud services.

[0024] In some aspects, the one or more processors are further configured to perform operations including generating a node access policy, the node access policy specifying an access policy for granting combined access of one or more containers on a node to a group of compute instances, and the one or more processors are further configured to perform operations including storing the container access policy in the memory.

[0025] In some aspects, the one or more processors are further configured to perform operations including granting access permissions equal to the combined access of the one or more containers assigned to the node.

[0026] In some aspects, the one or more processors are further configured to perform operations including dividing the compute instances having a plurality of compute instances into groups of nodes, each of the nodes having separate access, and the one or more processors are further configured to perform operations including assigning one or more containers to nodes having sufficient access based at least in part on the node access policy.

[0027] In some aspects, node access is predetermined and container access within each node is dynamically set.

[0028] In some aspects, the one or more processors are further configured to perform operations including testing access requirements of a cloud system; detecting that a particular application failed to access the one or more cloud services based at least in part on entries in the request log; and modifying permissions of a computing instance to correct the failure.

[0029] Other features and advantages of the disclosed embodiments will be realized by reference to the remaining portions of the specification, including the drawings and claims. Further features and advantages, as well as the structure and operation of various examples of the present disclosure, are described in detail below with reference to the accompanying drawings, where like reference numbers may indicate identical or functionally similar elements.

[0030] These and other embodiments are described in more detail below. For example, other embodiments are directed to systems, devices, and computer-readable media associated with the methods described herein.

[0031] The nature and advantages of the embodiments of the present disclosure may be better understood with reference to the following detailed description and accompanying drawings. [Brief description of the drawings]

[0032] [Figure 1] FIG. 1 illustrates a logical view of an exemplary cloud network architecture. [Diagram 2] FIG. 1 illustrates a logical configuration of a second exemplary cloud network architecture. [Diagram 3] FIG. 1 is a flowchart of an example process associated with a technique for automatically setting minimum cloud service access rights for a container application. [Figure 4] FIG. 1 is an exemplary block diagram illustrating an authorized configuration of the disclosed system. [Diagram 5] FIG. 1 is an exemplary block diagram illustrating prohibited configurations of the disclosed system. [Figure 6] FIG. 13 is an exemplary diagram of permitted groupings. [Figure 7] FIG. 1 is a simplified flowchart diagram of a technique for selective container access to a hosting node-based cloud service that uses a request forwarder as a service. [Figure 8] FIG. 2 is a block diagram illustrating one pattern for implementing a cloud infrastructure as a service system in accordance with at least one embodiment. [Figure 9] FIG. 2 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system in accordance with at least one embodiment. [Figure 10] FIG. 2 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system in accordance with at least one embodiment. [Figure 11] FIG. 2 is a block diagram illustrating another pattern for implementing a cloud infrastructure as a service system in accordance with at least one embodiment. [Figure 12] FIG. 1 is a block diagram illustrating an exemplary computer system according to at least one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0033] Detailed Description I. Introduction Modern computing frameworks abstract the concept of a machine. A machine can be considered simply a processing unit. Modern applications are increasingly built using containers, which are microservices packaged with their dependencies and configuration. Container management / clustering services are software for deploying and managing those containers at scale. As applications grow and span multiple containers deployed across multiple servers, their operation becomes more complex. To manage this complexity, container management / clustering services provide open source application programming interfaces (APIs) that control how and where those containers run. Container management / clustering services orchestrate clusters of virtual machines and schedule containers to run on those virtual machines based on available compute resources and the resource requirements of each container. Containers are grouped into pods, which are the basic operating unit of the container management / clustering service, and those pods scale to the desired state. Container management / clustering services also automatically manage service discovery, incorporate load balancing, track resource allocation, scale based on compute utilization, check the health of individual resources, and enable apps to self-heal by automatically restarting or replicating containers.

[0034] A network orchestrator can be used to allocate the number of instances of each container. Based on how busy each machine is, containers can be added or removed to increase workload density, resulting in cost savings. Containers can be customized for high input / output (I / O) bandwidth, improving performance.

[0035] Cloud providers can issue an identity for each compute node. That way, a node can have its own identity. There are standard mechanisms to get that identity. So if a container is running a process on a node, the system can call, for example, a specific IP address. This can return a specific identity ephemeral credential that is associated with that specific instance. That credential will have multiple parameters, not just what the specific machine is, but also what group of machines that machine belongs to, etc.

[0036] In one example, a customer can consume two services from a cloud vendor (e.g., compute and object storage). The customer's applications run on the compute instances and store and extract data from the object storage bucket. Without an integrated authentication mechanism, the customer is forced to operate in the same manner as in the pre-cloud legacy environment: the customer must create a user with their account, grant the user access to the object storage, and provide the user's credentials to the application. This last step can be particularly problematic. The credentials must be placed on the system before the application can be operational. When the system is rebooted or a new machine is added, this step must be repeated. This also must be performed manually by a technician who has access to the credentials, increasing the risk of the credentials being exposed. To make the process less cumbersome, the customer may decide to store the credentials persistently on the machine itself or in a location where multiple machines have access to the credentials. This may further increase the risk of the credentials being leaked. Apart from the security issues, such a process does not scale for complex and large workloads. The requirement for synchronous actions (e.g. credential creation, credential distribution) poses serious problems for highly automated cloud workloads, and the process must be repeated every time a customer wants to change their credentials.

[0037] Modern cloud vendors offer highly integrated Identity Access Management (IAM) solutions. IAM is a framework of policies and technologies that ensures that the right people in an enterprise have the right access to technology resources. IAM systems fall under the umbrella of IT security and data management. Identity and access management systems identify, authenticate, and authorize not only the individuals who will be utilizing IT resources, but also the hardware and applications that employees need to access. Identity and access management solutions have become more prevalent and important in recent years as regulatory compliance requirements become increasingly stringent and complex.

[0038] This addresses the need to ensure appropriate access to resources across increasingly heterogeneous technology environments and meet increasingly stringent compliance requirements. Compute instances, and other resources dedicated to hosting customer workloads, such as serverless functions, have their own identities recognized by cloud IAM under the customer's account. Customers may directly address these components as first-class principals, group them, and grant them the access they need. Compute machines, functions, and other components are automatically provisioned with short-lived credentials accessible to the processes they host. Applications extract those credentials and use them to access cloud resources.

[0039] Problems can arise when customers try to host multiple heterogeneous processes on a single compute instance. This is often the case when using compute nodes to run containers that are managed by orchestration frameworks such as those used by commercial container management / clustering services. The core of this problem is that the smallest granularity of identity is a single machine. Just because containers are hosted on the same machine does not mean that they are intended to have the same access to cloud resources. However, because a single machine is the most granular level of identity, all containers must share it.

[0040] Consider the following example: A customer consumes two services from a cloud vendor: compute and object storage. The customer's applications run on the compute instances and perform storage and extract data operations from buckets in the object storage. Without a unified authentication mechanism, the customer would be forced to operate in the same manner as in the pre-cloud legacy environment: the pre-cloud system would (1) create a user under the user's account, (2) grant the user access to the object storage, and (3) provide the user's credentials to the application.

[0041] This last step is particularly problematic: the credentials must be on the system before the application can be operational. When the system is rebooted or new machines are added, this step must be repeated. This must also be performed manually by a technician with access to the credentials, increasing the risk of the credentials being exposed. To save time, customers may decide to store the credentials permanently on the machine itself, or in a location where multiple machines have access to them, further increasing the risk of the credentials being leaked.

[0042] Apart from security issues, this process does not scale for large or complex workloads. The requirement for synchronous actions (e.g., credential creation or credential distribution) can pose problems for highly automated cloud workloads. Furthermore, this process must be repeated every time a customer wants to change their credentials.

[0043] Compute instances and other resources dedicated to hosting customer workloads, such as serverless functions, have their own identities recognized by cloud IAM under the customer's account. Customers can directly address these components as first-class principals, group them, and grant them the access they need. Compute instances, functions, and other components are automatically provisioned with short-lived credentials that are accessible to the processes they host. Applications can then extract those credentials and use them to access cloud resources.

[0044] In a typical implementation, a compute instance includes a locally accessible network service, commonly referred to as a metadata service. This service allows local workloads on the system to access various information propagated by the cloud vendor, such as the credentials of the principal representing the compute instance. The metadata service may only be accessible to processes running on this particular machine. The process makes a request to the metadata service, receives the credentials, and uses them to access other cloud services. These other services use the cloud vendor's IAM service to validate that the principal representing the instance has access to perform the requested operation.

[0045] Certain embodiments of the present disclosure may provide a method, system, and computer-readable medium for managing access to cloud-based services. Two types of access control are described herein. First, there is a container's access to a particular cloud service. This access may be determined based on the policy under which the container invokes the service, which the container has permission to or does not have permission to. Furthermore, the second type of access control is by a compute instance to a cloud service. Thus, a process running on a container needs permission for the container to access a cloud service, but also the compute instance of which the container is a part needs permission to have access to that cloud service.

[0046] This disclosure describes systems and techniques that broker requests between containers and cloud services to provide an appropriate level of access control. The disclosed techniques combine cloud orchestrator internal authentication to identify the container caller with cloud service authentication to authenticate calls made to the cloud service. In some examples, individual containers running on the system may not have direct access to metadata services or instance credentials. Instead, containers may send requests through a request forwarder component, which establishes the container identity and verifies that a particular container has access to communicate with the target service. The request forwarder component uses the instance credentials to authenticate calls to the target service. Compute instances may be configured to prevent containers from accessing metadata services. This technique effectively prevents processes within the container from using the compute instance credentials. As a result of these techniques, containers gain access to cloud services through the machine on which they are running, since direct access is not provided to the container.

[0047] When a container is initialized, it may be provided with credentials by a container orchestrator. This process may take a variety of forms, but the most common process results in the credentials being stored in the container's file system. According to one aspect of the present disclosure, a process running within a container intends to make a call to a cloud service. The process may send a request for the cloud service to a request forwarder. The request may include container credentials. The request forwarder may receive the request and determine the identity of the container by sending the container credentials to the container orchestrator. The request forwarder may verify that the container is authorized to access the target cloud service by referencing one or more policies stored in the system. The request forwarder may obtain instance credentials from a metadata service. The request forwarder may send a request including the compute instance credentials to the target cloud service. The cloud service may validate the request against cloud policies to verify that the instance is authorized by the cloud service to perform a given operation.

[0048] In this disclosure, an instance (e.g., a compute instance) is a hosted server running in either a Customer Enclave (publicly available) or a Service Enclave. If the instance has direct access to the hardware on which it runs, the instance can be considered a bare metal instance. If there is a hypervisor between the instance and the operating system, the instance can be considered a virtual instance. A hypervisor is software that allows a user to create one or more virtual machines and run them simultaneously. Hypervisors are also known as Virtual Machine Monitors (VMMs). One of the main features that a hypervisor provides is isolation, which means that if a guest crashes, it cannot affect the operation of the host or other guests. Hypervisors can be of two types: type 1 and type 2. Type 1 hypervisors are sometimes known as native hypervisors or bare metal hypervisors, and this type of hypervisor runs directly on top of the physical hardware. Each virtual operating system runs on the hypervisor. Examples of bare metal hypervisors can be Oracle VM server, VMware ESX / ESXi, and Microsoft Hyper-V. Type 2 hypervisors are sometimes known as hosted hypervisors. This type of hypervisor is installed as a software application on top of an existing host operating system (OS). Examples of hosted hypervisors can be Oracle VirtualBox, Microsoft Virtual PC, Vmware Server and Workstation.

[0049] In this disclosure, a container is a collection of schemas, objects, and related structures within a multitenant container database (CDB) that appears to an application as a logically separate database. Within a CDB, each container has a unique ID and name. The root and all pluggable databases (PDBs) are considered containers. PDBs separate data and operations so that from a user or application perspective, each PDB appears as if it were a traditional non-CDB.

[0050] FIG. 1 illustrates a logical view of an exemplary cloud network architecture 100. The cloud network architecture may include one or more containers (e.g., container 1 104, container 2 106, and container 3 108). In one implementation, the compute instance 102 may include a locally accessible network service, generally referred to as a metadata service 110. This metadata service 110 allows local workloads on the system to access various information propagated by the cloud vendor, including the credentials of the principal representing the compute instance 102. The metadata service 110 may only be accessible to processes running on this particular machine. The process contacts the metadata service 110 to obtain the credentials and can use the credentials to access other cloud services (e.g., cloud service A 112, cloud service B 114, or cloud service C 116). The service uses the cloud vendor's IAM service to verify that the principal representing the instance has access to perform the requested cloud operation.

[0051] Problems can arise when a customer seeks to host multiple heterogeneous processes on a single compute instance 102. This is often the case when compute nodes are used to run containers managed by an orchestration framework (e.g., container management / clustering services, etc.). The smallest granularity of identity may be a single machine. Previous techniques may have only assigned access to one or more cloud services to this machine, but not necessarily to different containers on the same machine. Just because containers are hosted on the same machine does not mean that they are intended to have the same access to cloud resources. However, since a single machine is the most granular level of identity, all containers must share it.

[0052] For example, as shown in FIG. 1, Container 1 104 communicates only with Cloud Service A 112; Container 2 106 communicates with Cloud Service A 112 and Cloud Service B 114; and Container 3 108 communicates with Cloud Service C 116. In one configuration, a customer may accept the risk that a node may have access to cloud services that were not intended for it. In this configuration, a compute node is granted all access that any container it hosts may need, and all containers receive access node credentials. This approach effectively reduces the level of security for convenience. For example, in the configuration shown in FIG. 1, compute instance 102 may be provided with access to Cloud Service A 112, Cloud Service B 114, and Cloud Service C 116. As an example, a process running on Container 1 104 may potentially be prohibited from accessing Cloud Service C 116 because the service belongs to a competing entity. If access were controlled only by compute instance, a process running in Container 1 104 could mistakenly gain access to Cloud Service C 116 because compute instance 102 provided permission for all containers to access all three cloud services shown.

[0053] In a second configuration, a customer can selectively disable metadata service 110 access to some containers. In this configuration, containers that do not need access to cloud resources can be barred from accessing metadata service 110. Although this can be simple to implement, it is an all-or-nothing approach. This configuration is useless for containers that have different access needs. Thus, a user may decide to only allow this container to access its metadata service across the network using firewall rules or network policies. Moreover, this only works if the container needs access to some service. Thus, by blocking the path through which the container obtains credentials, it is possible to at least isolate components that do not need access to cloud services.

[0054] In a third configuration, customers can distribute containers across separate machines. In this configuration, customers can create groups of compute nodes and give each group different access. The customer can then configure the container orchestrator to place containers on nodes with access that matches the needs of each container. Nodes can be split into different groups. The groups are assigned different access to their nodes, and then the scheduler in the container orchestrator is configured to simply assign specific containers to specific nodes. While this approach works in limited cases, the more you need to do it, the more fragmented the set of machines becomes, users lose machines, and overall workload density decreases. This configuration provides containers with adapted access to cloud resources. However, this configuration can be complex, requires careful planning and mapping of policies to nodes and nodes to containers, and can result in lower resource utilization.

[0055] Customers may use any combination of these options, increasing the complexity of access management. The drawbacks of the above solutions increase with the size and complexity of the workload, especially with the number of different containers and their cloud service access needs. Accepting the risk, the access given to individual containers becomes greater and greater, even when that access may not be necessary. Distributing a large number of containers may increase the number of different machines required, increasing the complexity of finding the correct set of access required on each node. In extreme cases, customers may decide to have a pool of machines per container type. This provides perfect access restrictions, but defeats the very purpose of using a container management service and significantly increases costs due to low process density.

[0056] Some providers, such as commercial container management / clustering services, offer support for containers (e.g., pods) to have their own type of principal in the managed services. So access can be granted directly to the container itself, rather than relying on the metadata service 110. While this may be a useful solution, it is limited to a given vendor and that vendor's specific services. Customers who wish to manage their own clusters or use a different framework will not be able to implement this solution.

[0057] II. Logical structure of cloud network architecture As cloud services add and remove containers or as rules are added, multiple types of instances may be required, which may grant different levels of access to different instances. This process may be difficult to perform at scale and is more complicated than a simple manual mapping of the network.

[0058] Machine-to-cloud service access can be managed by cloud IAM. The disclosed technique can determine two policies: one policy governs container access within a node; and a second policy governs node access within a cloud.

[0059] A potential IAM solution could establish a request forwarder that can identify the components making cloud service calls. The components can then use their identity to make calls to cloud provider services. Such a request forwarder can determine the configuration and have access to one or more policies that can determine which services can be called, with which APIs, and with which credentials.

[0060] Such a system can be difficult to implement at scale or in today's fast-paced software delivery paradigm. New or updated components are delivered daily, and each may have different needs for communicating with cloud services. This makes manually managing individual component policies impractical and error-prone. The result may be overly permissive policies that allow all components access to unnecessary services. Access to the compute nodes themselves must also be managed to ensure there are no conflicts between the compute nodes and the combined permissions of all containers running on them.

[0061] A second problem may be the management of compute node access. The above discussion assumes that a compute node has enough access to handle all requests from the various containers that reside on the node. However, managing compute node access can also be very difficult, especially in a rapidly changing environment. In principle, a compute node should have access that is the union of all access nodes of all containers hosted on that node. As containers on a node are added, removed, or rearranged, the access requirements of the node change. As a result of this, compute nodes are often granted excessive privileges as a means to ensure that all containers have the access they need. Best security practices dictate that node access should be granted to a minimal set. Managing node access has recently come to be properly understood as distinct from managing container access, and has become a major operational hurdle.

[0062] This disclosure describes a system that automatically infers a minimum set of access needs for both containers and compute nodes. Those needs are further expressed as policies that are provided to the request forwarder and cloud IAM. It also combines the lifecycle of the components with the lifecycle of the policies themselves. Furthermore, the system allows for the specification of criteria or access rules to control how access is assigned to nodes. Thus, by aggregating requests in a request log, the system can determine a minimum set of access needs that are used to determine the access requirements (e.g., a minimum set) of the container. In other words, the access requirements may be equal to a minimum set for the container (and / or the container's compute nodes).

[0063] Such a system allows users to provide new components and rapidly modify existing components while ensuring that the access granted to cloud services remains minimal. Without an automatic mechanism for generating strict and specific policies, users may be inclined to grant overly permissive access to containers, or more likely, to compute instances. Alternatively, if a high security level is expected, manually provisioning a centralized policy may have a negative impact on the performance of feature release cycles. The disclosed system allows development teams to achieve high speed delivery without sacrificing high security with access control. Furthermore, since the rules of access can be specified, the system can automatically partition containers into nodes and automatically build secure isolation boundaries for containers.

[0064] FIG. 2 shows a logical view of a second cloud network architecture 200. The disclosed system operates in two main phases. In the first phase, containers undergo integration testing. A test environment is provisioned with a request forwarder that is set to run in permissive mode. Such tests, typical of continuous delivery and containerized systems, exercise the container's functionality and trigger calls to the cloud provider. The requests to the cloud provider are logged by the request forwarder. This log is then aggregated to create cloud service access policies specific to a particular container and for the compute node (often a group of nodes) that hosts the container. The creation of the policies can be fine-tuned by access rules provided as system configuration. Such access rules may, for example, prevent one node from having access to two specific cloud services. The policies are then packaged together with the container in one deployment package.

[0065] A deployment package consists of a container, container-specific policies, and node policies. The infrastructure orchestration system deploys node access policies 224 to the cloud IAM 226, and the container and container-specific access policies 222 are deployed to the container cluster. The cluster then deploys the container and sends the policy to the request forwarder. Similarly, when a container is deleted, the container access in the forwarder and the unnecessary parts of the policy attached to the compute node are deleted.

[0066] The second cloud network architecture 200 may include a test environment 202. The test environment 202 may include one or more compute instances 204. The compute instances 204 may include one or more containers (e.g., container 1 206, container 2 208, and container 3 210). Each container may receive credentials to identify the container on the network. In various examples, a container orchestrator may provide an identity to the container when the container is created. The container identity may be stored in the container's memory.

[0067] As described above in FIG. 1, a process running within a container of a compute instance 204 can request a cloud service (e.g., cloud service A 228, cloud service B 230, or cloud service C 232). The container can send its container credentials to the request forwarder 214. The request forwarder 214 can act as a proxy between the container and the cloud service. The request forwarder 214 is responsible for obtaining the credentials so that components on the machine can call the cloud service with the instance credentials, thereby allowing the network architecture to disconnect all of those containers from accessing the credentials. Thus, those containers do not have the credentials and the containers must make all calls from now on through the request forwarder 214, as that is the only way to obtain the credentials. The request forwarder 214 can receive requests from the containers that include the container credentials. The request forwarder 214 can validate the credentials with the container orchestrator in the control plane. The request forwarder 214 can send container credentials to the container orchestrator and receive container identities back.

[0068] In the test environment 202, the request forwarder 214 can send a request to a metadata service. The request can include container credentials. The metadata service 212 can use the container credentials to obtain instance credentials. The request forwarder 214 can send requests for cloud services from one or more containers to a request log 216. The requests can be stored in the request log 216. The data in the request log 216 can be stored on a server. The aggregator 220 can aggregate one or more requests and store the aggregated list on a server. The aggregator 220 can also receive one or more access rules 218. The aggregator 220 can generate one or more container access policies 222. The aggregator 220 can generate one or more node access policies 224.

[0069] The production environment 252 may include a compute instance 254. The compute instance 254 may include one or more containers (e.g., container 1 256, container 2 258, and container 3 260). Each container may receive a credential to identify the container on the network. In various examples, a container orchestrator may provide an identity to the container when it is created. The container identity may be stored in the container's memory. The compute instance 254 may include a request forwarder 274. The request forwarder 274 may act as a proxy between the container and the cloud service. The request forwarder 274 is responsible for obtaining the credential so that components on the machine can call the cloud service with the instance credential, thereby allowing the network architecture to disconnect all of those containers from access to that credential. Thus, those containers do not have that credential and the container must make all calls from now on through the request forwarder 274, as that is the only way to obtain the credential. The request forwarder 274 can receive requests from containers that include container credentials. The request forwarder 274 can verify the credentials with a container orchestrator in the control plane. The request forwarder 274 can send the container credentials to the container orchestrator and receive the container identity back.

[0070] The compute instance 254 may include a metadata service 262. The request forwarder 274 may send a request to the metadata service 262. The request may include container credentials. The metadata service 262 may use the container credentials to obtain the instance credentials. The request forwarder 274 may have access to container access policies 222. In various embodiments, the container access policies 222 may be stored in the compute instance 254. In a production environment, the request forwarder 274 may receive one or more container access policies 222. The container access policies 222 may be used to determine which cloud services, if any, the container is authorized to use. The request forwarder 274 may use the container access policies 222 to determine the container's permissions. The request forwarder 274 may request the instance credentials from the metadata server 222. The metadata server 262 may send the instance credential information based on the container's container access policies 222. The request forwarder 274 can send the request to a cloud service (e.g., cloud service A 228, cloud service B 230, or cloud service C 232). The instance credentials can be attached to the request sent to the cloud service.

[0071] 3 is a flow chart of an example process 300 associated with a technique for automatically setting minimum cloud service access rights for a container application. At 302, the process 300 can begin by adding a new or modified container to the cloud server infrastructure.

[0072] At 304, the container can enter a test environment. In the test environment, the request forwarder can be set to run in permissive mode. The test environment does not control access, but is used to identify access needs. Such tests, which may be typical for continuous delivery and containerized systems, exercise the container's functionality and trigger calls to the cloud provider. The test environment should be as comprehensive as possible, otherwise there may be gaps in the testing and request needs may not be captured and therefore access will not be granted.

[0073] A test environment may exist in the cloud. Within the test environment, there may be a baseline policy that provides the maximum permitted access that a system can have. For example, the baseline policy may provide restrictions such as connecting to features as a service (e.g., due to security concerns). So in this case, a test machine would not be provided with access to features as a service. So, if you try to connect through that system, if the policy is set to allow it otherwise, the test will fail because the machine used for testing will not have that access. This is so that the developer can see what access is not possible.

[0074] At 306, test results can be received. The test results can determine whether one or more applications running on the container can access one or more cloud services. If the test results are negative, the process does not proceed because the container is faulty or incorrectly coded, or because a baseline policy prohibits access.

[0075] At 308, the request to the cloud provider may be logged in a request log by the request forwarder. The request log may be stored on the server.

[0076] At 310, the request logs are then aggregated to create cloud service access policies specific to a particular container and for the compute node (often a group of nodes) that hosts the container. This process can be performed by a type of infrastructure orchestration. The creation of the policies can be fine-tuned by access rules provided as system configurations. Such access rules may, for example, prevent one node from having access to two specific cloud services. The policies are then packaged together with the container into one deployment package.

[0077] A deployment package can be generated at 312. The deployment package can include containers, container-specific policies, and node policies.

[0078] At 314, an infrastructure orchestration module can receive the deployment package.

[0079] At 316, the infrastructure orchestration module can deploy the node access policy to the cloud IAM. The node access policy provides machine access.

[0080] At 318, the infrastructure orchestration module can deploy the container and the container-specific access policies are deployed to the container cluster.

[0081] At 320, the container cluster can deploy the container, where the container goes to a container cluster or container that has a policy, which itself is deployed to the container, and the policy is deployed to the request forwarder, so the request forwarder knows what access individual containers running on the node can use, and controls that access, and the cloud IAM has the policy to allow those machines to use that access based on their configuration.

[0082] The container cluster sends the policy to the requesting forwarder at 322. Similarly, when a container is deleted, the container access in the forwarder and the unnecessary parts of the policy attached to the compute node are deleted.

[0083] In the example operation, a new container is identified that is being used for the first time in an application. In order to proceed, two things are needed. First, the process needs a policy that allows that container to use the object store so that the node knows about this container but not other containers. This allows the container to access the object store, and the machine on which this container will run needs to have access to the object store. So those two policies are created and packaged with the new container. Then, when the container is deployed and infrastructure orchestration unpacks it, the deployable artifact provision provides access to the cloud IAM, so that the machine and this container will be deployed to a group of machines that now have access to the object store that they did not have access to before. Furthermore, when that container is deployed to a container cluster, the request forwarder, or other mechanism that controls such access, will know that this container has access to the object store. So even though there may be other containers running on that machine, they will not get access to the object store, even though the machine technically has access to the object store.

[0084] A container cluster can be distributed across multiple machines or multiple compute instances (e.g., Kubernetes, Docker Swarm, or Openshift). A container orchestrator can be a system that places and manages containers on multiple machines. Those machines can be abstracted as workers. Those machines can be added or removed by the container orchestrator, which provides the container to another machine if a machine goes down. Those machines can have groups (e.g., a group of machines), which can be a set of machines on which this container can run. This group of machines can also be called a node from the container orchestrator's perspective. From the cloud infrastructure perspective, it can be called an instance. An instance can be a virtual machine or a bare metal computer.

[0085] Cloud IAM can control access to groups of either virtual machines or bare metal machines or a combination that a container cluster sets up. Cloud IAM controls which principals in that cloud environment can access what in the cloud. Machines are typically organized into groups and those groups of machines are granted access to cloud services. Groups allow for scaling up and down to grant access to groups of machines so that if one instance cannot handle the load another machine can take over.

[0086] FIG. 4 illustrates an example configuration diagram 400 showing permitted configurations of the disclosed system. In this example, a workload consisting of six containers can be assumed, and the following connectivity needs were identified in a test environment: Table 1 illustrates a container / service or resource chart. An example of a service can be Oracle Cloud Infrastructure (OCI), which allows for network management and scaling. Another example of a service can be VMware Configuration Manager (VCM). A resource within a service can be a subnet or a load balancer.

[0087] [Table 1]

[0088] In this example, access rules can be specified by a system administrator to reflect the security requirements of the services. An exemplary access rule can require access to Service C / Resource X separate from Service C / Resource Y. An access rule can also require access to Service A separate from Service D. These rules provide restrictions on what access can be granted to compute nodes and regulate system operation. Based on known communication paths, and access control restrictions, the system can identify which containers can be co-located on the same node. This is illustrated in the "Allowed Configuration" graph shown in FIG. 4. Using graph operations, the system can identify groups of containers that can be shared and the number of distinct nodes. With reference to FIG. 4, Container 1 402 can be co-located with Container 2 404, Container 3 406, Container 4 408, or Container 5 410, but cannot be co-located with Container 6 412. Container 2 can be co-located with Container 1 402, Container 3 406, Container 4 408, Container 5 410, or Container 6 412. Container 3 406 can be collocated with Container 1 402, Container 2 404, Container 4 408, or Container 5 410. Container 4 408 can be collocated with Container 1 402, Container 2 404, Container 3 406, or Container 6 412. Container 5 can be collocated with Container 1 402, Container 2 404, or Container 3 406. Container 6 can be collocated with Container 2 404 or Container 4 408.

[0089] FIG. 5 illustrates an example configuration diagram 500 showing prohibited configurations of the disclosed system. FIG. 5 follows the same access rules as specified in Table 1 and described in FIG. 4. One approach to perform this operation is to use a graph to capture prohibited configurations of containers. This is illustrated in the "prohibited configurations" graph shown in FIG. 5. Given the prohibited configurations, any set of disconnected graph nodes is a valid grouping of containers. This is captured in the "allowed groupings" as shown in FIG. 4. It should be noted that since container 2 504 is disconnected from all nodes, it could be included in any container grouping. It should also be noted that there are other possible configurations that are not shown. The system only provides strict criteria for container groupings, allowing the container orchestrator to schedule containers within these constraints based on its own algorithms as well as considering other factors. As shown in FIG. 5, container 1 502 is prohibited from being co-located with container 6 512. container 2 504 is not prohibited from being co-located with any container. Container 6 512 is prohibited from co-locating with Container 1 502, Container 3 506, or Container 5 510. Container 3 506 is prohibited from co-locating with Container 6 512. Container 5 510 is prohibited from co-locating with Container 6 512 and Container 4 508. Container 4 508 is prohibited from co-locating with Container 5 510.

[0090] From the above "permitted grouping" example, the access needs of the node group that represents the union of the access needs of the containers can be identified and scheduled as shown in Table 2.

[0091] [Table 2]

[0092] FIG. 6 shows an example diagram 600 of permitted groupings. The groupings follow the predefined access rules. The groupings also provide natural separation of containers resulting from the rules. The access needs of individual containers and node groups are captured in respective policies and packaged with the respective containers. A cloud orchestration system such as Terraform may be used to automatically deploy the policies to cloud IAM and perform the required node groupings using mechanisms specific to the particular cloud infrastructure. If the system operates in a mode where no compute node policies are set, the system may look at existing node groups, identify node groups that match the container, and then report insufficient or excessive permissions.

[0093] 6 shows node group 1 602 and node group 2 604. Node group 1 602 may include container 2 608, container 4 612, and container 6 616. Node group 2 may include container 1 606, container 3 610, container 5 614, and container 2 608.

[0094] The permitted groups diagram 600 is from an implementation perspective. The system can analyze the diagram 600 to determine permitted configurations. For example, a first group of machines can host containers 2, 6, and 4, while a second group can host containers 1, 3, 5, and 2. As shown, container 2 can be hosted on both group 1 and group 2 of machines, which can be useful for performance reasons. For example, there may be excess capacity in node group 2, and container 2 can be placed there. This is just one possibility.

[0095] In one example, every container could get its own node group. This would work, but it's suboptimal and useless. The system knows that this is an allowed grouping of containers, so it can create node groups, it can schedule containers within those node groups, it can identify the combined policies for each group because it knows what the specific needs of the containers are, and it can assume that the nodes themselves don't need access.

[0096] But in general, the combined needs of a group of machines are just the needs of all the containers on that machine. So node group 1 gets access to service BDC resource X node group 2 gets access to service ABC resource Y. This results in a policy. By identifying the needs of the containers, the system creates logical separation within the application that respects those rules. Logical separation of the application can be created by putting containers on groups of nodes and putting those containers on those groups of nodes, just like a security engineer would do in a review with a team. The system can infer logical separation of the environment through those rules that it wants to achieve, through a completely different mechanism. And those rules will hold, for example, if the system keeps adding containers that have other access needs, etc.

[0097] There may be situations where this problem cannot be solved graphically. This is when one container violates some of the rules. If the system separates access to service C resource X from service resource Y, the system will have a container that has access to both resources in the service. Since both service resources cannot be accessed, the problem cannot be solved and the system will fail. This will result in a failure at the aggregation step, which is probably beneficial. For this reason, the system will decide that it cannot actually proceed because the designer has created a container that violates the rules the system has established.

[0098] FIG. 7 is a flowchart of an example process 700 associated with a technique for automatically setting minimum cloud service access rights for a container application. In some implementations, one or more process blocks of FIG. 7 may be performed by a computer system (e.g., computer system 1200 as shown in FIG. 12). Additionally or alternatively, one or more process blocks of FIG. 7 may be performed by one or more components of device 1200, such as processing unit 1204, storage subsystem 1218, communication subsystem 1224, I / O subsystem 1208, and bus subsystem 1202. Processing unit 1204 may include sub-processing units 1232, 1234. Storage subsystem 1218 may include system memory 1210. System memory 120 may include application programs 1212, program data 1214, and operating system 1216. Storage subsystem 1218 may include computer-readable storage medium reader 1220 and computer-readable storage medium 1222.

[0099] 7, process 700 may include receiving one or more requests to access one or more cloud services (block 710). For example, as described above, a computer system may receive one or more requests to access one or more cloud services. The requests may be received from one or more of the same or different sources.

[0100] 7, the process 700 may include saving the one or more requests in a request log (block 720). For example, as described above, the computer system may save the one or more requests in a request log.

[0101] 7, the process 700 may include receiving one or more access rules applicable to the cloud service access rights (block 730). For example, as described above, the computer system may receive one or more access rules applicable to the cloud service access rights.

[0102] 7, process 700 may include aggregating one or more requests from the request log to determine access requirements for a container, the container configured to store one or more applications (block 740). For example, as described above, a computer system may aggregate one or more requests from the request log to determine access requirements for a container, the container configured to store one or more applications.

[0103] 7, process 700 may include generating a container access policy that defines access of the container and one or more cloud services, the container access policy based at least in part on the aggregated one or more requests and the one or more access rules (block 750). For example, as described above, the computer system may generate a container access policy that defines access of the container and one or more cloud services, the container access policy based at least in part on the aggregated one or more requests and the one or more access rules.

[0104] 7, the process 700 may include storing the container access policy in a memory (block 760). For example, as described above, a computer system may store the container access policy in a memory.

[0105] 7, process 700 may include sending the container access policy to a request forwarder of a compute instance in the production environment, where the request forwarder accesses the container access policy to grant the container access to one or more cloud services (block 770). For example, as described above, the computer system may send the container access policy to a request forwarder of a compute instance in the production environment, where the request forwarder accesses the container access policy to grant the container access to one or more cloud services.

[0106] Process 700 may include additional implementations, such as any single implementation or any combination of implementations described below and / or in conjunction with one or more other processes described elsewhere herein.

[0107] Process 700 may operate in two modes. In a first mode, process 700 controls access to cloud services. In a first node, both container access and node access are dynamically set by the system. All access requirements for containers and nodes used by the cluster are automatically set. The system may be configured to adjust how access is granted to nodes. In the simplest case, all nodes are equal and any container can be scheduled on any node. In this case, each node needs access permissions equal to the combined access of all containers. However, it is common for application designers to isolate containers so that an attack on one container (i.e., a container with high exposure or low trust) does not expose other containers in the same node. The container isolation process may be a manual process and depends on an expert's understanding of the system and security constraints. In the disclosed system, the requirement to determine how individual containers should be isolated may be removed. The system may be configured with general rules on how access should be granted (e.g., one system should not have access to two specific services or resources). Once such rules are set, the system will automatically split the cluster into groups of nodes with separate access and assign containers to the correct nodes with sufficient access. At the same time, the system still has the flexibility to assign containers that can work in multiple groups to any node in the cluster. Although this mechanism has advantages, some users may strongly prefer to manually configure the access of nodes in the system or may not be willing to leave AM access management to an automated system.

[0108] In a first implementation, process 700 includes generating a node access policy, where the node access policy specifies an access policy for granting combined access of one or more containers on the node to a group of compute instances, and process 700 further includes storing the container access policy in a memory.

[0109] In a second implementation, alone or in combination with the first implementation, the process 700 includes granting access permissions equal to the combined access of one or more containers assigned to the node.

[0110] In a third implementation, alone or in combination with one or more of the first and second implementations, process 700 includes dividing a computing instance having a plurality of computing instances into a group of nodes, each of the nodes having separate access, and process 700 further includes assigning one or more containers to nodes having sufficient access based at least in part on a node access policy.

[0111] In the second mode, the process 700 does not control node access to the cloud service. In a fourth implementation, alone or in combination with one or more of the first to third implementations, the access is predefined and the container access in each node is dynamically set. If the user does not want to leave the access management to an automated system, the node's access may be predefined and known to the system. Here, only the container access granted in the node is dynamically set. The system may then place a container with specific access needs on a node with sufficient access, as needed. Since the access granted to the compute nodes is predefined and cannot be changed by the system, a container may require access that no node in the cluster can grant. This can be detected during the testing phase. If the forwarder fails to make a request, the failure is logged and an indication is given that the compute instance's permissions must be changed. There are multiple ways to determine if the forwarder's request is failing. In a test environment, assuming that the test and production instances have identical permissions, the failure of the test itself is sufficient evidence.

[0112] In a fifth implementation, alone or in combination with one or more of the first through fourth implementations, the process 700 includes testing access requirements of a cloud system, detecting that a particular application failed to access one or more cloud services based at least in part on an entry in the request log, and modifying permissions of the compute instance to correct the failure. In some examples, the entry in the request log is a previously stored request. For example, the request log may be configured to store multiple requests. And, each request in the request log may be considered an entry.

[0113] In a sixth implementation example, alone or in combination with one or more of the first through fifth implementation examples, process 700 includes setting a request forwarder for each container type in the production environment to an allow mode, where the allow mode grants access to one or more cloud services for one or more applications stored in the container, and process 700 further includes receiving one or more requests from the request forwarder and, in response to a number of the one or more requests exceeding a threshold requirement, switching the request forwarder to a restrictive mode, where the restrictive mode grants access to the one or more cloud services for the one or more applications based in part on the container access policies.

[0114] Although Figure 7 illustrates example blocks of process 700, in some implementations, process 700 may include additional blocks, fewer blocks, different blocks, or blocks arranged differently than those shown in Figure 7. Additionally or alternatively, two or more of the blocks of process 700 may be performed in parallel.

[0115] III. Centralized Request Forwarders A separate centralized node, or a separate set of request forwarders supporting multiple nodes, or any combination of nodes are all permissible. The disclosed system can support all of these options by managing container policies at the forwarder assigned to the container and node policies for the node that hosts that forwarder, regardless of where that node is located. Similarly, the system may support a centralized cloud version of a request forwarder by granting access to entities based on combined container needs.

[0116] There can be multiple ways to locate the request forwarder. In one example, the request forwarder can be centrally located. In another example, the request forwarder can be external to the machine as long as there is a mechanism to provision its policy and identify the container. In some examples, there may be no request forwarder in the production model. Finally, the container policy can be obtained in the production environment. This can be done if the client believes that the testing is not comprehensive enough or if a test environment does not exist. In this case, the technique may initially run only in the production environment with an allow mode, learning the access needs as more requests are received. Once the access needs are identified, the technique can obtain enough information to know all the access needs and begin to fulfill those needs.

[0117] In one example, there are no new requests over the last few requests. There can be various thresholds that can be set to indicate that sufficient access has been achieved and identified. The system then begins to enforce access rights. The system can initially run with an overly permissive policy, but eventually the policy becomes more restrictive. This is preferable to constantly running in permissive mode.

[0118] Access can be removed when a container is deleted. When a new container is introduced into the system, access may not be available, and the process understands the container lifecycle, e.g., when a new version of a container is deployed, it will remove previously established rules.

[0119] The cloud network architecture may include multiple compute instances. One or more compute instances may store various containers. Another separate compute instance may include a request forwarder, a metadata service, and one or more policies. The cloud network architecture may include various cloud services (e.g., cloud service A, cloud service B, and cloud service C). The cloud network architecture may also include a container orchestrator. The request forwarder may be centralized to multiple nodes, or each node may have its own request forwarder. A centralized / dedicated request forwarder node implies that different nodes (which may include multiple containers) can forward requests to cloud services via the centralized node. Instance credentials do not need to be stored on all nodes, only on the node where the request forwarder resides. A request forwarder does not necessarily need to be on the same node as the container it supports.

[0120] There are advantages and disadvantages to both of these architectures. With regard to a centralized / dedicated request forwarder, the node's resources can be dedicated only to fulfilling the request forwarder duties, while other nodes can focus on using their resources to execute tasks. Apart from this, the instance credentials do not need to be stored on each node, but only on the centralized instance. However, a disadvantage of this approach is that the centralized request forwarder can be a single point of failure. If it goes down, none of the containers will be able to make requests to the cloud service. With regard to having a request forwarder on each node, it is much simpler to create and make the network rules / architecture work. Another advantage of a per-node forwarder is that each node has its own request forwarder, and as a result, each node only handles the workload of its containers. One disadvantage of this design is that per-node credentials must be maintained to access the cloud service. This makes the problem of proper key rotation and revocation as difficult as the number of nodes involved. A hybrid approach may involve using a subset of the cluster nodes to host the request forwarders.

[0121] In various embodiments, there are no request forwarders in production. This system can be useful in situations where there are no request forwarders in production. In this case, there is no policy to set individual containers if the container has all the access that the node has. However, the node's access will be constrained to the union of all containers on the node. This does not restrict container access to a minimum, but it still automatically restricts node access, which is a significant advantage over manually setting that access.

[0122] In various embodiments, container policies can be captured in a production environment. This system will most likely be used in two stages, where the data needed to build the policies is captured in a trusted test environment and then deployed along with the containers to the production system, although alternative mechanisms using only the production environment are possible.

[0123] This mechanism involves switching the request forwarder's allow / restrict mode separately for each container type in the production environment. When a container is deployed for the first time, the request forwarder sends all requests to the cloud service while creating a log of the requests. After enough data is collected, it switches to the restrict mode for that particular container. This implementation does not provide the same level of security since there is a period during which all access to the cloud service is allowed. However, it reduces the cost of system operation and may be sufficient for some users, especially if container turnaround is low. The decision that enough data has been obtained to switch modes can be made manually, based on time, or by analysis of how many new types of requests are being forwarded. For example, if all requests sent to the forwarder do not include new requests given a specified number of requests, the collected data may be considered sufficient. Access to nodes is managed in a similar way, starting with excessive access (within reasonable limits set by the administrator) and reducing to the specific permissions required by each individual container once the access needs of all containers on the node have been fully established.

[0124] IV. Infrastructure as a Service (IAAS) As mentioned above, infrastructure as a service (IaaS) is one particular type of cloud computing. IaaS may be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, a cloud computing provider may host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, an IaaS provider may also provide various services (e.g., billing, monitoring, logging, security, load balancing, clustering, etc.) that accompany those infrastructure components. Thus, these services may be policy-driven, so that an IaaS user may be able to implement policies that drive load balancing to maintain application availability and performance.

[0125] In some examples, an IaaS customer may access resources and services over a wide area network (WAN), such as the Internet, and can utilize the cloud provider's services to install the remaining elements of the application stack. For example, a user can log into an IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create storage buckets for workloads and backups, and even install enterprise software on the VMs. The customer can then utilize the provider's services to perform a variety of functions, including balancing network traffic, troubleshooting application issues, monitoring performance, managing disaster recovery, and more.

[0126] In most cases, the cloud computing model will require the participation of a cloud provider, which may or may not be a third-party service dedicated to providing (e.g., offering, renting, selling) IaaS, or an entity may choose to deploy a private cloud and become its own provider of infrastructure services.

[0127] In some examples, IaaS deployment is the process of putting a new application or a new version of an application onto a prepared application server, etc. It may also include the process of preparing the server (e.g., installing libraries, daemons, etc.). This is often managed below the hypervisor layer (e.g., server, storage, network hardware, and virtualization) by the cloud provider. Thus, the customer may be responsible for handling (e.g., on self-service virtual machines (e.g., that may be spun up on demand), middleware, and / or application deployment, etc.

[0128] In some instances, IaaS provisioning may refer to obtaining the computers or virtual hosts to be used and even installing the required libraries or services on them. In most cases, deployment does not include provisioning, which may need to be performed first.

[0129] In some cases, there are two distinct challenges in IaaS provisioning. First, there is the initial challenge of provisioning an initial set of infrastructure before anything works. Second, there is the challenge of evolving the existing infrastructure after everything has been provisioned (e.g., adding new services, modifying services, removing services, etc.). In some cases, these two challenges can be addressed by allowing the configuration of the infrastructure to be defined declaratively. In other words, the infrastructure (e.g., which components are needed and how they interact) can be defined by one or more configuration files. Thus, the overall topology of the infrastructure (e.g., which resources depend on which resources and how they each work together) can be described declaratively. In some examples, once the topology is defined, workflows can be generated that create and / or manage the different components described in the configuration files.

[0130] In some examples, the infrastructure may have many interconnected elements. For example, there may be one or more virtual private clouds (VPCs) (e.g., a potentially on-demand pool of configurable and / or shared computing resources), also known as a core network. In some examples, there may also be one or more security group rules and one or more virtual machines (VMs) provisioned to define how the security of the network is set up. Other infrastructure elements such as load balancers, databases, etc. may also be provisioned. The infrastructure may evolve incrementally as more infrastructure elements are desired and / or added.

[0131] In some examples, continuous deployment techniques may be employed to enable deployment of infrastructure code across various virtual computing environments. In addition, the described techniques may enable infrastructure management within these environments. In some examples, a service team may write code that is desired to be deployed to one or more, but often many, different production environments (e.g., across a wide variety of geographic locations, sometimes spanning the entire world). However, in some examples, the infrastructure onto which the code will be deployed must first be set up. In some examples, provisioning may be done manually, and a provisioning tool may be utilized to provision the resources, and / or once the infrastructure is provisioned, a deployment tool may be utilized to deploy the code.

[0132] FIG. 8 is a block diagram 800 illustrating an example pattern of an IaaS architecture according to at least one embodiment. A service operator 802 may be communicatively coupled to a secure host tenancy 804, which may include a virtual cloud network (VCN) 806 and a secure host subnet 808. In some examples, the service operator 802 may employ one or more client computing devices. The client computing devices may be portable handheld devices (e.g., iPhone®, mobile phones, iPad®, computing tablets, personal digital assistants (PDAs)) or wearable devices (e.g., Google Glass® head-mounted displays) running software such as Microsoft Windows Mobile® and / or various mobile operating systems such as iOS, Windows Phone, Android, BlackBerry 8, Palm OS, and supporting Internet, email, short message service (SMS), Blackberry®, or other communication protocols. Alternatively, the client computing devices may be general-purpose personal computers, including, by way of example, personal computers and / or laptop computers running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux® operating systems. The client computing device may be a workstation computer running any of a variety of commercially available UNIX or UNIX-like operating systems, including, but not limited to, various GNU / Linux operating systems such as Google Chrome OS.Alternatively or additionally, the client computing devices may be other electronic devices capable of communicating over the VCN 806 and / or an Internet-accessible network, such as thin-client computers, Internet-enabled gaming systems (e.g., Microsoft Xbox game consoles with or without a Kinect® gesture input device), and / or personal messaging devices.

[0133] The VCN 806 may include a local peering gateway (LPG) 810. The LPG 810 may be communicatively coupled to a secure shell (SSH) VCN 812 via an LPG 810 included in the SSH VCN 812. The SSH VCN 812 may include an SSH subnet 814, which may be communicatively coupled to a control plane VCN 816 via an LPG 810 included in the control plane VCN 816. The SSH VCN 812 may also be communicatively coupled to a data plane VCN 818 via the LPG 810. The control plane VCN 816 and the data plane VCN 818 may be included in a service tenancy 819, which may be owned and / or operated by the IaaS provider.

[0134] The control plane VCN 816 may include a control plane de-militarized zone (DMZ) tier 820 that serves as a perimeter network (e.g., a portion of an enterprise network between an enterprise intranet and an external network). DMZ-based servers may have limited responsibility and help mitigate security breaches. In addition, the DMZ tier 820 may include one or more load balancer (LB) subnets 822, a control plane app tier 824 that may include an app subnet 826, and a control plane data tier 828 that may include a database (DB) subnet 830 (e.g., a front-end DB subnet and / or a back-end DB subnet). The LB subnet 822 included in the control plane DMZ tier 820 may be communicatively coupled to the app subnet 826 included in the control plane app tier 824 and an Internet gateway 834 that may be included in the control plane VCN 816, and the app subnet 826 may be communicatively coupled to the DB subnet 830, a service gateway 836, and a network address translation (NAT) gateway 838 included in the control plane data tier 828. The control plane VCN 816 may include a service gateway 836 and a NAT gateway 838.

[0135] The control plane VCN 816 may include a data plane mirror app layer 840, which may include an app subnet 826. The app subnet 826 included in the data plane mirror app layer 840 may include a virtual network interface controller (VNIC) 842, which may run a compute instance 844. The compute instance 844 may communicatively couple the app subnet 826 of the data plane mirror app layer 840 to the app subnet 826, which may be included in the data plane app layer 846.

[0136] The data plane VCN 818 may include a data plane app layer 846, a data plane DMZ layer 848, and a data plane data layer 850. The data plane DMZ layer 848 may include a LB subnet 822, which may be communicatively coupled to an app subnet 826 of the data plane app layer 846 and an Internet gateway 834 of the data plane VCN 818. The app subnet 826 may be communicatively coupled to a service gateway 836 of the data plane VCN 818 and a NAT gateway 838 of the data plane VCN 818. The data plane data layer 850 may also include a DB subnet 830, which may be communicatively coupled to the app subnet 826 of the data plane app layer 846.

[0137] The Internet gateways 834 of the control plane VCNs 816 and data plane VCNs 818 may be communicatively coupled to a metadata management service 852, which may be communicatively coupled to the public Internet 854. The public Internet 854 may be communicatively coupled to a NAT gateway 838 of the control plane VCNs 816 and data plane VCNs 818. The service gateways 836 of the control plane VCNs 816 and data plane VCNs 818 may be communicatively coupled to cloud services 856.

[0138] In some examples, a service gateway 836 in a control plane VCN 816 or a data plane VCN 818 may make an application programming interface (API) call to a cloud service 856 without traversing the public Internet 854. An API call from a service gateway 836 to a cloud service 856 may be one-way. That is, the service gateway 836 may make an API call to a cloud service 856, and the cloud service 856 may send the requested data to the service gateway 836. However, the cloud service 856 may not initiate the API call to the service gateway 836.

[0139] In some examples, secure host tenancy 804 may be directly connected to service tenancy 819, which may otherwise be separate. Secure host subnet 808 may communicate with SSH subnet 814 through LPG 810, which may allow bidirectional communication through otherwise separate systems. Connecting secure host subnet 808 to SSH subnet 814 may give secure host subnet 808 access to other entities in service tenancy 819.

[0140] The control plane VCN 816 may enable users of the service tenancy 819 to set up or otherwise provision desired resources. The desired resources provisioned in the control plane VCN 816 may be deployed or otherwise used in the data plane VCN 818. In some examples, the control plane VCN 816 may be separate from the data plane VCN 818, and the data plane mirror app layer 840 of the control plane VCN 816 may communicate with the data plane app layer 846 of the data plane VCN 818 via a VNIC 842, which may be included in the data plane mirror app layer 840 and the data plane app layer 846.

[0141] In some examples, a user or customer of the system may make a request, such as, for example, a create, read, update, or delete (CRUD) operation, through the public Internet 854, which may communicate the request to a metadata management service 852. The metadata management service 852 may communicate the request to the control plane VCN 816 through an Internet gateway 834. The request may be received by a LB subnet 822 included in the control plane DMZ layer 820. The LB subnet 822 may determine that the request is valid, and in response to this determination, the LB subnet 822 may send the request to an app subnet 826 included in the control plane app layer 824. If the request is validated and requires a call to the public Internet 854, the call to the public Internet 854 may be sent to a NAT gateway 838, which may make the call to the public Internet 854. Memory that may be desired to be stored by the request may be stored in the DB subnet 830.

[0142] In some examples, the data plane mirror app layer 840 may facilitate direct communication between the control plane VCN 816 and the data plane VCN 818. For example, it may be desired that a configuration change, update, or other appropriate modification be applied to resources included in the data plane VCN 818. Through the VNIC 842, the control plane VCN 816 may communicate directly with resources included in the data plane VCN 818, thereby performing the change, update, or other appropriate modification to the configuration.

[0143] In some embodiments, the control plane VCN 816 and the data plane VCN 818 may be included in the service tenancy 819. In this case, a user or customer of the system may not own or operate either the control plane VCN 816 or the data plane VCN 818. Instead, an IaaS provider may own or operate the control plane VCN 816 and the data plane VCN 818, both of which may be included in the service tenancy 819. This embodiment may allow for network isolation that may prevent a user or customer from interacting with other users' or other customers' resources. This embodiment may also allow a user or customer of the system to store databases informally without having to rely on the public Internet 854 for storage, which may not have the desired level of security.

[0144] In another embodiment, the LB subnet 822 included in the control plane VCN 816 may be configured to receive signals from the service gateway 836. In this embodiment, the control plane VCN 816 and the data plane VCN 818 may be configured to be called by the IaaS provider's customers without calling the public Internet 854. The IaaS provider's customers may desire this embodiment because databases used by the customers may be controlled by the IaaS provider and may be stored on the service tenancy 819, which may be isolated from the public Internet 854.

[0145] 9 is a block diagram 900 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 902 (e.g., service operator 802 of FIG. 8 ) may be communicatively coupled to a secure host tenancy 904 (e.g., secure host tenancy 804 of FIG. 8 ), which may include a virtual cloud network (VCN) 906 (e.g., VCN 806 of FIG. 8 ) and a secure host subnet 908 (e.g., secure host subnet 808 of FIG. 8 ). The VCN 906 may include a local peering gateway (LPG) 910 (e.g., LPG 810 of FIG. 8 ), which may be communicatively coupled to a secure shell (SSH) VCN 912 (e.g., SSH VCN 812 of FIG. 6 ) via an LPG 810 included in the SSH VCN 912. SSH VCN 912 may include an SSH subnet 914 (e.g., SSH subnet 814 in FIG. 8 ), which may be communicatively coupled to a control plane VCN 916 (e.g., control plane VCN 816 in FIG. 8 ) via an LPG 910 included in the control plane VCN 916. The control plane VCN 916 may be included in a service tenancy 919 (e.g., service tenancy 819 in FIG. 8 ), and the data plane VCN 918 (e.g., data plane VCN 818 in FIG. 8 ) may be included in a customer tenancy 921, which may be owned or operated by a user or customer of the system.

[0146] The control plane VCN 916 may include a control plane DMZ tier 920 (e.g., control plane DMZ tier 820 of FIG. 8 ) that may include a LB subnet 922 (e.g., LB subnet 822 of FIG. 8 ), a control plane app tier 924 (e.g., control plane app tier 824 of FIG. 8 ) that may include an app subnet 926 (e.g., app subnet 826 of FIG. 8 ), and a control plane data tier 928 (e.g., control plane data tier 828 of FIG. 8 ) that may include a database (DB) subnet 930 (e.g., similar to DB subnet 830 of FIG. 8 ). The LB subnet 922 included in the control plane DMZ layer 920 may be communicatively coupled to an app subnet 926 included in the control plane app layer 924 and to an Internet gateway 934 (e.g., Internet gateway 834 in FIG. 8 ) that may be included in the control plane VCN 916, and the app subnet 926 may be communicatively coupled to a DB subnet 930 included in the control plane data layer 928, to a service gateway 936 (e.g., service gateway in FIG. 8 ), and to a network address translation (NAT) gateway 938 (e.g., NAT gateway 838 in FIG. 8 ). The control plane VCN 916 may include the service gateway 936 and the NAT gateway 938.

[0147] The control plane VCN 916 may include a data plane mirror app layer 940 (e.g., data plane mirror app layer 840 of FIG. 8 ), which may include an app subnet 926. The app subnet 926 included in the data plane mirror app layer 940 may include a virtual network interface controller (VNIC) 942 (e.g., VNIC 842) that may run a compute instance 944 (e.g., similar to compute instance 844 of FIG. 8 ). The compute instance 944 may facilitate communication between the app subnet 926 of the data plane mirror app layer 940 and the app subnet 926 that may be included in the data plane app layer 946 (e.g., data plane app layer 846 of FIG. 8 ) via the VNIC 942 included in the data plane mirror app layer 940 and the VNIC 942 included in the data plane app layer 946.

[0148] An internet gateway 934 included in the control plane VCN 916 may be communicatively coupled to a metadata management service 952 (e.g., metadata management service 852 of FIG. 8), which may be communicatively coupled to a public internet 954 (e.g., public internet 854 of FIG. 8). The public internet 954 may be communicatively coupled to a NAT gateway 938 included in the control plane VCN 916. A service gateway 936 included in the control plane VCN 916 may be communicatively coupled to cloud services 956 (e.g., cloud services 856 of FIG. 8).

[0149] In some examples, the data plane VCN 918 may be included in the customer tenancy 921. In this case, the IaaS provider may provide a control plane VCN 916 for each customer, and the IaaS provider may set up a unique compute instance 944 included in the service tenancy 919 for each customer. Each compute instance 944 may enable communication between the control plane VCN 916 included in the service tenancy 919 and the data plane VCN 918 included in the customer tenancy 921. The compute instance 944 may enable resources provisioned in the control plane VCN 916 included in the service tenancy 919 to be deployed or otherwise used in the data plane VCN 918 included in the customer tenancy 921.

[0150] In another example, an IaaS provider customer may have a database that resides in customer tenancy 921. In this example, control plane VCN 916 may include a data plane mirror app layer 940 that may include app subnet 926. The data plane mirror app layer 940 may reside in data plane VCN 918, but the data plane mirror app layer 940 may not reside in data plane VCN 918. That is, the data plane mirror app layer 940 may have access to customer tenancy 921, but the data plane mirror app layer 940 may not reside in data plane VCN 918 or be owned or operated by the IaaS provider customer. The data plane mirror app layer 940 may be configured to make calls to the data plane VCN 918, but may not be configured to make calls to any entities included in the control plane VCN 916. A customer may desire to deploy or otherwise use resources in a data plane VCN 918 that are provisioned in the control plane VCN 916, and the data plane mirror app layer 940 may facilitate the desired deployment or other use of the customer's resources.

[0151] In some embodiments, the IaaS provider's customer may apply filters to the data plane VCN 918. In this embodiment, the customer may determine what the data plane VCN 918 can access, and the customer may restrict access from the data plane VCN 918 to the public Internet 954. The IaaS provider may not be able to filter or otherwise control access of the data plane VCN 918 to any external networks or databases. Filters and controls applied by the customer on the data plane VCN 918 included in the customer tenancy 921 can help isolate the data plane VCN 918 from other customers and the public Internet 954.

[0152] In some embodiments, cloud services 956 may be invoked by the service gateway 936 to access services that may not be on the public internet 954, on the control plane VCN 916, or on the data plane VCN 918. The connection between the cloud services 956 and the control plane VCN 916 or the data plane VCN 918 may not be live or continuous. The cloud services 956 may be on different networks owned or operated by the IaaS provider. The cloud services 956 may be configured to receive calls from the service gateway 936 and may be configured not to receive calls from the public internet 954. Some cloud services 956 may be isolated from other cloud services 956, and the control plane VCN 916 may be isolated from cloud services 956 that may not be in the same region as the control plane VCN 916. For example, the control plane VCN 916 may be located in “Region 1” and cloud service “Deployment 6” may be located in Region 1 and “Region 2.” If a call to Deployment 6 is made by a service gateway 936 included in a control plane VCN 916 located in Region 1, the call may be sent to Deployment 6 in Region 1. In this example, the control plane VCN 916, or Deployment 6 in Region 1, may not be communicatively coupled to or otherwise in communication with Deployment 6 in Region 2.

[0153] 10 is a block diagram 1000 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 1002 (e.g., service operator 802 of FIG. 8 ) may be communicatively coupled to a secure host tenancy 1004 (e.g., secure host tenancy 804 of FIG. 8 ), which may include a virtual cloud network (VCN) 1006 (e.g., VCN 806 of FIG. 8 ) and a secure host subnet 1008 (e.g., secure host subnet 808 of FIG. 8 ). The VCN 1006 may include an LPG 1010 (e.g., LPG 810 of FIG. 8 ), which may be communicatively coupled to an SSH VCN 1012 (e.g., SSH VCN 812 of FIG. 8 ) via an LPG 1010 included in the SSH VCN 1012. SSH VCN 1012 may include an SSH subnet 1014 (e.g., SSH subnet 814 in FIG. 8 ), which may be communicatively coupled to a control plane VCN 1016 (e.g., control plane VCN 816 in FIG. 8 ) via an LPG 1010 included in the control plane VCN 1016, and to a data plane VCN 1018 (e.g., data plane 818 in FIG. 8 ) via an LPG 1010 included in the data plane VCN 1018. The control plane VCN 1016 and the data plane VCN 1018 may be included in a service tenancy 1019 (e.g., service tenancy 819 in FIG. 8 ).

[0154] The control plane VCN 1016 may include a control plane DMZ tier 1020 (e.g., control plane DMZ tier 820 of FIG. 8 ) that may include a load balancer (LB) subnet 1022 (e.g., LB subnet 822 of FIG. 8 ), a control plane app tier 1024 (e.g., control plane app tier 824 of FIG. 8 ) that may include an app subnet 1026 (e.g., similar to app subnet 1026 of FIG. 8 ), and a control plane data tier 1028 (e.g., control plane data tier 828 of FIG. 8 ) that may include a DB subnet 1030. The LB subnet 1022 included in the control plane DMZ layer 1020 may be communicatively coupled to an app subnet 1026 included in the control plane app layer 1024 and to an Internet gateway 1034 (e.g., Internet gateway 834 in FIG. 8 ) that may be included in the control plane VCN 1016, and the app subnet 1026 may be communicatively coupled to a DB subnet 1030 included in the control plane data layer 1028 and to a service gateway 1036 (e.g., service gateway in FIG. 8 ) and a network address translation (NAT) gateway 1038 (e.g., NAT gateway 838 in FIG. 8 ). The control plane VCN 1016 may include the service gateway 1036 and the NAT gateway 1038.

[0155] The data plane VCN 1018 may include a data plane app layer 1046 (e.g., data plane app layer 846 of FIG. 8 ), a data plane DMZ layer 1048 (e.g., data plane DMZ layer 848 of FIG. 8 ), and a data plane data layer 1050 (e.g., data plane data layer 850 of FIG. 8 ). The data plane DMZ layer 1048 may include a LB subnetwork 1022 that may be communicatively coupled to a trusted app subnetwork 1060 and an untrusted app subnetwork 1062 of the data plane app layer 1046 and to an Internet gateway 1034 included in the data plane VCN 1018. The trusted app subnetwork 1060 may be communicatively coupled to a service gateway 1036 included in the data plane VCN 1018, a NAT gateway 1038 included in the data plane VCN 1018, and a DB subnetwork 1030 included in the data plane data layer 1050. The untrusted app subnet 1062 may be communicatively coupled to a service gateway 1036 included in the data plane VCN 1018 and a DB subnet 1030 included in the data plane data layer 1050. The data plane data layer 1050 may include a DB subnet 1030 that may be communicatively coupled to a service gateway 1036 included in the data plane VCN 1018.

[0156] The untrusted app subnet 1062 may include one or more primary VNICs 1064(1)-(N), which may be communicatively coupled to tenant virtual machines (VMs) 1066(1)-(N). Each tenant VM 1066(1)-(N) may be communicatively coupled to a respective app subnet 1067(1)-(N), which may be included in a respective container egress VCN 1068(1)-(N), which may be included in a respective customer tenancy 1070(1)-(N). Each secondary VNIC 1072(1)-(N) may facilitate communication between the untrusted app subnet 1062 included in the data plane VCN 1018 and the app subnet included in the container egress VCN 1068(1)-(N). Each container egress VCN 1068(1)-(N) may include a NAT gateway 1038, which may be communicatively coupled to the public Internet 1054 (e.g., public Internet 854 of FIG. 8 ).

[0157] An Internet gateway 1034 included in the control plane VCN 1016 and in the data plane VCN 1018 may be communicatively coupled to a metadata management service 1052 (e.g., metadata management system 852 of FIG. 8 ), which may be communicatively coupled to the public Internet 1054. The public Internet 1054 may be communicatively coupled to a NAT gateway 1038 included in the control plane VCN 1016 and in the data plane VCN 1018. A service gateway 1036 included in the control plane VCN 1016 and in the data plane VCN 1018 may be communicatively coupled to cloud services 1056.

[0158] In some embodiments, the data plane VCN 1018 may be integrated with a customer tenancy 1070. This integration may be useful or desirable for an IaaS provider's customer in some cases, such as when run-time support of code is desired. A customer may provide code to be executed that may be disruptive, may communicate with other customer resources, or may otherwise cause undesirable effects. In response, the IaaS provider may determine whether to execute the code that the customer provided to the IaaS provider.

[0159] In some examples, a customer of an IaaS provider may request to grant temporary network access to the IaaS provider to attach a function to the data plane layer app 1046. The code to perform the function may be executed in the VMs 1066(1)-(N), and the code may not be configured to run anywhere else on the data plane VCN 1018. Each VM 1066(1)-(N) may be connected to one customer tenancy 1070. Each container 1071(1)-(N) included in the VMs 1066(1)-(N) may be configured to execute code. In this case, there may be double isolation (e.g., the containers 1071(1)-(N) may execute code, and the containers 1071(1)-(N) may be included in at least the VMs 1066(1)-(N) included in the untrusted app subnet 1062), which may help prevent erroneous or otherwise undesirable code from damaging the IaaS provider's network or from damaging a different customer's network. The containers 1071(1)-(N) may be communicatively coupled to the customer tenancy 1070 and may be configured to send or receive data from the customer tenancy 1070. The containers 1071(1)-(N) may not be configured to send or receive data from any other entities in the data plane VCN 1018. Once the code execution is complete, the IaaS provider may kill or otherwise discard the containers 1071(1)-(N).

[0160] In some embodiments, the trusted app subnet 1060 may execute code that may be owned or operated by the IaaS provider. In this embodiment, the trusted app subnet 1060 may be communicatively coupled to the DB subnet 1030 and may be configured to perform CRUD operations on the DB subnet 1030. The untrusted app subnet 1062 may be communicatively coupled to the DB subnet 1030, but in this embodiment, the untrusted app subnet may be configured to perform read operations on the DB subnet 1030. The containers 1071(1)-(N) that may be included in each customer's VMs 1066(1)-(N) and that may execute code from the customer may not be communicatively coupled to the DB subnet 1030.

[0161] In other embodiments, the control plane VCN 1016 and the data plane VCN 1018 may not be directly communicatively coupled. In this embodiment, there may be no direct communication between the control plane VCN 1016 and the data plane VCN 1018. However, communication may occur indirectly through at least one method. An LPG 1010 may be established by an IaaS provider that may facilitate communication between the control plane VCN 1016 and the data plane VCN 1018. In another example, the control plane VCN 1016 or the data plane VCN 1018 may make a call to a cloud service 1056 through a service gateway 1036. For example, a call from the control plane VCN 1016 to the cloud service 1056 may include a request for a service that may communicate with the data plane VCN 1018.

[0162] 11 is a block diagram 1100 illustrating another example pattern of an IaaS architecture, according to at least one embodiment. A service operator 1102 (e.g., service operator 802 of FIG. 8 ) may be communicatively coupled to a secure host tenancy 1104 (e.g., secure host tenancy 804 of FIG. 8 ), which may include a virtual cloud network (VCN) 1106 (e.g., VCN 806 of FIG. 8 ) and a secure host subnet 1108 (e.g., secure host subnet 808 of FIG. 8 ). VCN 1106 may include an LPG 1110 (e.g., LPG 810 of FIG. 8 ), which may be communicatively coupled to an SSH VCN 1112 (e.g., SSH VCN 812 of FIG. 8 ) via an LPG 1110 included in SSH VCN 1112. SSH VCN 1112 may include an SSH subnet 1114 (e.g., SSH subnet 814 in FIG. 8 ), which may be communicatively coupled to a control plane VCN 1116 (e.g., control plane VCN 816 in FIG. 8 ) via an LPG 1110 included in control plane VCN 1116, and to a data plane VCN 1118 (e.g., data plane 818 in FIG. 8 ) via an LPG 1110 included in data plane VCN 1118. The control plane VCN 1116 and the data plane VCN 1118 may be included in a service tenancy 1119 (e.g., service tenancy 819 in FIG. 8 ).

[0163] The control plane VCN 1116 may include a control plane DMZ layer 1120 (e.g., control plane DMZ layer 820 of FIG. 8 ) that may include a LB subnet 1122 (e.g., LB subnet 822 of FIG. 8 ), a control plane app layer 1124 (e.g., control plane app layer 824 of FIG. 8 ) that may include an app subnet 1126 (e.g., app subnet 826 of FIG. 8 ), and a control plane data layer 1128 (e.g., control plane data layer 828 of FIG. 8 ) that may include a DB subnet 1130 (e.g., DB subnet 830 of FIG. 8 ). The LB subnet 1122 included in the control plane DMZ layer 1120 may be communicatively coupled to an app subnet 1126 included in the control plane app layer 1124 and to an Internet gateway 1134 (e.g., Internet gateway 834 in FIG. 8 ) that may be included in the control plane VCN 1116, and the app subnet 1126 may be communicatively coupled to a DB subnet 1130 included in the control plane data layer 1128 and to a service gateway 1136 (e.g., service gateway in FIG. 8 ) and a network address translation (NAT) gateway 1138 (e.g., NAT gateway 838 in FIG. 8 ). The control plane VCN 1116 may include the service gateway 1136 and the NAT gateway 1138.

[0164] The data plane VCN 1118 may include a data plane app layer 1146 (e.g., data plane app layer 846 in FIG. 8 ), a data plane DMZ layer 1148 (e.g., data plane DMZ layer 848 in FIG. 8 ), and a data plane data layer 1150 (e.g., data plane data layer 850 in FIG. 8 ). The data plane DMZ layer 1148 may include a LB subnet 1122 that may be communicatively coupled to a trusted app subnet 1160 (e.g., trusted app subnet 860 in FIG. 8 ) and an untrusted app subnet 1162 (e.g., untrusted app subnet 862 in FIG. 8 ) of the data plane app layer 1146 and an Internet gateway 1134 included in the data plane VCN 1118. The trusted app subnet 1160 may be communicatively coupled to a service gateway 1136 included in the data plane VCN 1118, a NAT gateway 1138 included in the data plane VCN 1118, and a DB subnet 1130 included in the data plane data layer 1150. The untrusted app subnet 1162 may be communicatively coupled to a service gateway 1136 included in the data plane VCN 1118 and a DB subnet 1130 included in the data plane data layer 1150. The data plane data layer 1150 may include the DB subnet 1130, which may be communicatively coupled to a service gateway 1136 included in the data plane VCN 1118.

[0165] The untrusted app subnet 1162 may include primary VNICs 1164(1)-(N), which may be communicatively coupled to tenant virtual machines (VMs) 1166(1)-(N) residing within the untrusted app subnet 1162. Each tenant VM 1166(1)-(N) may execute code in a respective container 1167(1)-(N), which may be communicatively coupled to an app subnet 1126, which may be included in a data plane app tier 1146, which may be included in a container egress VCN 1168. Each secondary VNIC 1172(1)-(N) may facilitate communication between the untrusted app subnet 1162, which may be included in the data plane VCN 1118, and the app subnet, which may be included in the container egress VCN 1168. The container egress VCN may include a NAT gateway 1138, which may be communicatively coupled to the public Internet 1154 (e.g., public Internet 854 of FIG. 8 ).

[0166] An Internet gateway 1134 included in the control plane VCN 1116 and in the data plane VCN 1118 may be communicatively coupled to a metadata management service 1152 (e.g., metadata management system 852 of FIG. 8 ), which may be communicatively coupled to the public Internet 1154. The public Internet 1154 may be communicatively coupled to a NAT gateway 1138 included in the control plane VCN 1116 and in the data plane VCN 1118. A service gateway 1136 included in the control plane VCN 1116 and in the data plane VCN 1118 may be communicatively coupled to cloud services 1156.

[0167] In some examples, the pattern illustrated by the architecture of block diagram 1100 of FIG. 11 may be considered an exception to the pattern illustrated by the architecture of block diagram 800 of FIG. 8 and may be desirable for a customer of an IaaS provider when the IaaS provider cannot communicate directly with the customer (e.g., in a disconnected area). Each container 1167(1)-(N) contained in a VM 1166(1)-(N) for each customer may be accessed in real time by the customer. The containers 1167(1)-(N) may be configured to make calls to each secondary VNIC 1172(1)-(N) contained in the app subnet 1126 of the data plane app tier 1146, which may be contained in a container egress VCN 1168. The secondary VNIC 1172(1)-(N) may send the call to a NAT gateway 1138, which may send the call to the public Internet 1154. In this example, containers 1167(1)-(N) that may be accessed in real time by a customer may be isolated from the control plane VCN 1116 and may be isolated from other entities included in the data plane VCN 1118. Containers 1167(1)-(N) may also be isolated from resources from other customers.

[0168] In another example, a customer may invoke cloud service 1156 using container 1167(1)-(N). In this example, the customer may execute code in container 1167(1)-(N) that requests a service from cloud service 1156. Container 1167(1)-(N) may send the request to secondary VNIC 1172(1)-(N), which may send the request to a NAT gateway, which may send the request to public Internet 1154. Public Internet 1154 may send the request to LB subnet 1122, which is included in control plane VCN 1116, via Internet gateway 1134. In response to determining that the request is valid, LB subnet 1126 may send the request to app subnet 1126, which may send the request to cloud service 1156 via service gateway 1136.

[0169] It should be noted that the IaaS architectures 800, 900, 1000, 1100 depicted in the figures may have components other than those depicted. Additionally, the depicted embodiments are only some examples of cloud infrastructure systems that may incorporate embodiments of the present disclosure. In some other embodiments, the IaaS systems may have more or fewer components than depicted, may combine two or more components, or may have different configurations or arrangements of components.

[0170] In certain embodiments, the IaaS system described herein may include a suite of application, middleware, and database services products that are self-service, subscription-based, elastically scalable, reliable, highly available, and securely delivered to customers. One example of such an IaaS system is Oracle Cloud Infrastructure (OCI), offered by the assignee of the present application.

[0171] 12 illustrates an exemplary computer system 1200 on which various examples of the disclosure may be implemented. System 1200 may be used to implement any of the computer systems described above. As illustrated, computer system 1200 includes a processing unit 1204 that communicates with several peripheral subsystems via a bus subsystem 1202. These peripheral subsystems may include a processing acceleration unit 1206, an I / O subsystem 1208, a storage subsystem 1218, and a communication subsystem 1224. Storage subsystem 1218 includes a tangible computer-readable storage medium 1222 and a system memory 1210.

[0172] Bus subsystem 1202 provides a mechanism for allowing the various components and subsystems of computer system 1200 to communicate with each other as intended. Although bus subsystem 1202 is shown diagrammatically as a bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 1202 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. For example, such architectures may include a Peripheral Component Interconnect (PCI) bus, which may be implemented as an Industry Standard Architecture (ISA) bus, a MicroChannel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a mezzanine bus manufactured in accordance with the IEEE P1386.1 standard.

[0173] The processing unit 1204, which may be implemented as one or more integrated circuits (e.g., conventional microprocessors or microcontrollers), controls the operation of the computer system 1200. One or more processors may be included in the processing unit 1204. These processors may include single-core processors or multi-core processors. In particular embodiments, the processing unit 1204 may be implemented as one or more independent processing units 1232 and / or 1234, with a single-core processor or a multi-core processor included in each processing unit. In other embodiments, the processing unit 1204 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors into one chip.

[0174] In various examples, the processing unit 1204 may execute various programs in response to program code and may maintain multiple programs or processes running simultaneously. At any given time, some or all of the program code to be executed may reside in the processor 1204 and / or the storage subsystem 1218. Through appropriate programming, the processor 1204 may provide the various functions described above. The computer system 1200 may additionally include a processing acceleration unit 1206, which may include a digital signal processor (DSP), special purpose processor, and / or others.

[0175] The I / O subsystem 1208 may include user interface input devices and user interface output devices. User interface input devices may include a keyboard, a pointing device such as a mouse or trackball, a touchpad or touch screen integrated into a display, a scroll wheel, a click wheel, a dial, a button, a switch, a keypad, a voice input device with a voice command recognition system, a microphone, and other types of input devices. User interface input devices may include, for example, a motion sensing and / or gesture recognition device such as a Microsoft Kinect® motion sensor that allows a user to control and interact with an input device such as a Microsoft Xbox® 360 game controller through a natural user interface using gestures and spoken commands. User interface input devices may also include an eye gesture recognition device such as a Google Glass® blink detector that detects eye movements from a user (e.g., "blinking" during picture taking and / or menu selection) and translates the eye gestures as input to an input device (e.g., Google Glass®). Additionally, the user interface input devices may include a voice recognition sensing device that allows a user to interact with a voice recognition system (eg, the Siri® navigator) through voice commands.

[0176] User interface input devices may also include, but are not limited to, three-dimensional (3D) mice, joysticks or pointing sticks, game pads and graphic tablets, as well as audio / visual devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers 3D scanners, 3D printers, laser range finders, and eye-tracking devices. In addition, user interface input devices may include medical imaging input devices, such as, for example, computed tomography, magnetic resonance imaging, positron emission tomography, medical ultrasound, etc. User interface input devices may also include audio input devices, such as, for example, MIDI keyboards, digital musical instruments, etc.

[0177] User interface output devices may include a display subsystem, indicator lights, or non-visual displays such as audio output devices, etc. The display subsystem may be a flat panel device such as one using a cathode ray tube (CRT), liquid crystal display (LCD) or plasma display, a projection device, a touch screen, etc. In general, when the term "output device" is used, it is intended to include all possible types of devices and mechanisms for outputting information from computer system 1200 to a user or to another computer. For example, user interface output devices may include a variety of display devices that visually convey textual, graphical, and audio / visual information, such as, but not limited to, monitors, printers, speakers, headphones, car navigation systems, plotters, audio output devices, and modems.

[0178] Computer system 1200 may include a storage subsystem 1218 that includes software elements currently shown as being within system memory 1210. System memory 1210 may store program instructions loadable and executable on processing unit 1204, as well as data generated during the execution of these programs.

[0179] Depending on the configuration and type of computer system 1200, the system memory 1210 may be volatile (such as random access memory (RAM)) and / or non-volatile (such as read only memory (ROM), flash memory, etc.). RAM typically contains data and / or program modules that are immediately accessible to and / or currently being operated on and executed by the processing unit 1204. In some implementations, the system memory 1210 may include a number of different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some implementations, a basic input / output system (BIOS), containing the basic routines that help to transfer information between elements within the computer system 1200, such as during start-up, may typically be stored in ROM. By way of example and not limitation, the system memory 1210 also illustrates application programs 1212, which may include client applications, a web browser, a mid-tier application, a relational database management system (RDBMS), and the like, program data 1214, and an operating system 1216. By way of example, operating system 1216 may include various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux® operating systems, various commercially available UNIX® or UNIX-like operating systems (including but not limited to various GNU / Linux® operating systems, Google Chrome® OS, etc.), and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® 10 OS, and Palm® OS operating systems.

[0180] The storage subsystem 1218 may also provide a tangible computer-readable storage medium for storing the basic programming and data constructs that provide the functionality of some embodiments. Software (programs, code modules, instructions) that, when executed by a processor, provide the above-described functionality may be stored in the storage subsystem 1218. These software modules or instructions may be executed by the processing unit 1204. The storage subsystem 1218 may also provide a repository for storing data used in accordance with the present disclosure.

[0181] Storage subsystem 1200 may also include a computer readable storage medium reader 1220 that may be further connected to a computer readable storage medium 1222. Along with system memory 1210, and optionally in combination with system memory 1210, computer readable storage medium 1222 may comprehensively represent remote, local, fixed, and / or removable storage devices plus storage media for containing, storing, transmitting, and retrieving computer readable information on a temporary and / or more permanent basis.

[0182] The computer readable storage medium 1222 containing the code or portions of code may also include any suitable medium known or used in the art, including, but not limited to, storage media and communication media, such as volatile and non-volatile, removable and non-removable media, implemented in any manner or technology for storing and / or transmitting information. This may include tangible computer readable storage media, such as RAM, ROM, Electronically Erasable Programmable ROM (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD), or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage, or other tangible computer readable media. This may also include non-tangible computer readable media, such as a data signal, data transmission, or any other medium that can be used to transmit the desired information and that can be accessed by the computing system 1200.

[0183] By way of example, the computer readable storage medium 1222 may include hard disk drives that read from and write to non-removable, non-volatile magnetic media, magnetic disk drives that read from and write to removable, non-volatile magnetic disks, and optical disk drives that read from and write to removable, non-volatile optical disks, such as CD ROMs, DVDs, and Blu-Ray disks, or other optical media. The computer readable storage medium 1222 may include, but is not limited to, Zip drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVD disks, digital video tapes, and the like. The computer readable storage medium 1222 may also include flash memory-based SSDs, enterprise flash drives, solid state drives (SSDs) based on non-volatile memory such as solid state ROM, SSDs based on volatile memory such as solid state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs that use a combination of DRAM and flash memory-based SSDs. The disk drives and their associated computer-readable media may provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for the computer system 1200.

[0184] The communications subsystem 1224 provides an interface to other computer systems and networks. The communications subsystem 1224 serves as an interface for transmitting and receiving data between other systems and the computer system 1200. For example, the communications subsystem 1224 may enable the computer system 1200 to connect to one or more devices via the Internet. In some embodiments, the communications subsystem 1224 may include a radio frequency (RF) transceiver component for accessing wireless voice and / or data networks (e.g., using cellular technology, advanced data network technologies such as 3G, 4G, or EDGE (enhanced data rates for global evolution), Wi-Fi (IEEE 802.11 family of standards, or other mobile communications technologies, or any combination thereof), a global positioning system (GPS) receiver component, and / or other components. In some embodiments, the communications subsystem 1224 may provide wired network connectivity (e.g., Ethernet) in addition to or instead of a wireless interface.

[0185] In some embodiments, the communications subsystem 1224 may also receive incoming communications in the form of structured and / or unstructured data feeds 1226, event streams 1228, event updates 1230, etc., on behalf of one or more users who may be using the computer system 1200.

[0186] As an example, the communications subsystem 1224 may be configured to receive data feeds 1226 in real time from users of social networks and / or other communications services, such as web feeds such as Twitter® feeds, Facebook® updates, Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third party sources.

[0187] Additionally, the communications subsystem 1224 may also be configured to receive data in the form of continuous data streams, which may include event streams 1228 of real-time events and / or event updates 1230, which may be continuous or infinite in nature with no apparent end. Examples of applications that generate continuous data may include, for example, sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, and the like.

[0188] The communications subsystem 1224 may also be configured to output structured and / or unstructured data feeds 1226, event streams 1228, event updates 1230, etc. to one or more databases that may be in communication with one or more streaming data source computers coupled to the computer system 1200.

[0189] The computer system 1200 may be one of a variety of types, including a handheld portable device (e.g., an iPhone® mobile phone, an iPad® computing tablet, a PDA), a wearable device (e.g., a Google Glass® head mounted display), a PC, a workstation, a mainframe, a kiosk, a server rack, or any other data processing system.

[0190] Due to the ever-changing nature of computers and networks, the illustration of computer system 1200 shown in the figure is intended as only one specific example. Many other configurations are possible having more or fewer components than the system shown in the figure. For example, customized hardware may be used and / or particular elements may be implemented in hardware, firmware, software (including applets), or a combination. Additionally, connections to other computing devices, such as network input / output devices, may be employed. Based on the disclosure and teachings provided herein, one of ordinary skill in the art will appreciate other ways and / or methods for implementing the various examples.

[0191] Any of the software components or functions described herein may be implemented as software code executed by a processor using any suitable computer language, such as, for example, Java, C++, or Perl, using conventional or object-oriented techniques. The software code may be stored as a series of instructions or commands on a computer-readable medium for storage and / or transmission, suitable media including random access memory (RAM), read-only memory (ROM), magnetic media such as hard drives or floppy disks, or optical media such as compact disks (CDs) or DVDs (digital versatile disks), flash memory, etc. The computer-readable medium may also be any combination of such storage or transmission devices.

[0192] Such programs may also be encoded and transmitted using carrier signals adapted to be transmitted over wired, optical, and / or wireless networks conforming to various protocols, including the Internet. Thus, computer-readable media according to embodiments of the present disclosure may be created using data signals encoded with such programs. Computer-readable media encoded with program code may be packaged with a compatible device or provided separately from other devices (e.g., via Internet download). Any such computer-readable media may reside on or within a single computer program product (e.g., a hard drive or an entire computer system) or may be present on or within different computer program products within a system or network. The computer system may include a monitor, printer, or other suitable display for providing a user with any of the results mentioned herein.

[0193] Although specific embodiments of the present disclosure have been described, various modifications, alternatives, alternative configurations, and equivalents are also encompassed within the scope of the present disclosure. The embodiments of the present disclosure are not limited to operation in a particular specific data processing environment, but can freely operate in multiple data processing environments. In addition, although the embodiments of the present disclosure have been described using a particular sequence of transactions and steps, it should be apparent to one skilled in the art that the scope of the present disclosure is not limited to the sequence of transactions and steps described. Various features and aspects of the above-described embodiments may be used individually or jointly.

[0194] Furthermore, while embodiments of the present disclosure have been described using a particular combination of hardware and software, it should be appreciated that other combinations of hardware and software are within the scope of the present disclosure. Embodiments of the present disclosure may be implemented using only hardware, only software, or a combination thereof. The various processes described herein may be implemented on the same processor or on different processors in any combination. Thus, when a component or module is described as being configured to perform a particular operation, such configuration may be achieved, for example, by designing an electronic circuit to perform the operation, by programming a programmable electronic circuit (such as a microprocessor) to perform the operation, or by any combination thereof. Processes may communicate using a variety of techniques, including, but not limited to, conventional techniques for inter-process communication, and different pairs of processes may use different techniques, or the same pair of processes may use different techniques at different times.

[0195] The specification and drawings are therefore to be regarded as illustrative rather than restrictive. It will be apparent, however, that additions, subtractions, deletions, and other modifications and changes may be made thereto without departing from the broader spirit and scope of the appended claims. Thus, although specific embodiments of the disclosure have been described, they are not intended to be limiting. Various modifications and equivalents are intended to be within the scope of the appended claims.

[0196] The use of the words "a" and "an" and "the" and similar referents in the context of describing the disclosed embodiments (particularly in the context of the appended claims) should be construed to include both the singular and the plural, unless otherwise stated herein or clearly contradicted by context. The words "comprising," "having," "including," and "containing" should be construed as open-ended (i.e., meaning "including but not limited to"), unless otherwise stated. The word "connected" should be construed as partly or wholly contained, attached, or coupled, even if there is something intervening. The recitation of ranges of values ​​herein is merely intended as a shorthand way of referring individually to each individual value contained within the range, unless otherwise stated herein, and each individual value is incorporated herein as if it were individually set forth herein. All methods described herein can be performed in any suitable order, unless otherwise stated herein or clearly contradicted by context. Any examples provided herein, or the use of exemplary language (e.g., "such as"), are intended merely to better illustrate embodiments of the disclosure and do not limit the scope of the disclosure unless specifically claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the disclosure.

[0197] Disjunctive language, such as the phrase "at least one of X, Y, or Z," is intended to be understood within the context in which it is commonly used to indicate that an item, term, etc. can be either X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z), unless specifically stated otherwise. Thus, such disjunctive language is generally not intended to, and should not, imply that a particular embodiment requires that at least one of X, at least one of Y, or at least one of Z, respectively, be present.

[0198] Preferred embodiments of the present disclosure are described herein, including the best mode known to the inventors for carrying out the disclosure. Modifications of these preferred embodiments will become apparent to those skilled in the art upon reading the above description. The inventors anticipate that such modifications will be adopted by those skilled in the art as appropriate, and the inventors intend for the present disclosure to be carried out in ways other than as specifically described herein. Accordingly, this disclosure includes all modifications and equivalents of the subject matter recited in the claims appended hereto as permitted by applicable law. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the present disclosure unless otherwise indicated herein or otherwise clearly contradicted by context.

[0199] All references cited in this specification, including publications, patent applications, and patents, are hereby incorporated by reference to the same extent as if each individual reference was individually and specifically indicated to be incorporated by reference and was set forth in its entirety herein.

[0200] Although the above specification describes aspects of the disclosure with reference to specific embodiments thereof, those skilled in the art will recognize that the disclosure is not limited thereto. Various features and aspects of the above-described disclosure may be used individually or jointly. Moreover, the embodiments may be utilized in any number of environments and applications beyond those described herein without departing from the broader spirit and scope of the specification. Thus, the specification and drawings should be regarded as illustrative rather than restrictive.

[0201] The specific details of particular embodiments may be combined in any suitable manner or may be varied from those illustrated and described in the specification without departing from the spirit and scope of the embodiments of the present disclosure.

[0202] The above description of exemplary embodiments of the present disclosure has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the present disclosure to the precise form described, and numerous modifications and variations are possible in light of the above teachings. The embodiments have been selected to best explain the principles of the present disclosure and their practical application, thereby enabling those skilled in the art to best utilize the present disclosure with various modifications in various examples as suited to the particular applications contemplated.

[0203] The entire contents of all publications, patents, and patent applications cited herein are hereby incorporated by reference for all purposes.

Claims

1. 1. A method comprising: Receiving one or more requests to access one or more cloud services; storing the one or more requests in a request log; receiving one or more access rules applicable to cloud service access rights for the one or more cloud services; aggregating the one or more requests from the request log to determine access requirements for a container, the container configured to store one or more applications, the method further comprising: generating a container access policy defining access rights for a container to the one or more cloud services, the container access policy being generated based at least in part on the aggregated one or more requests and the one or more access rules, the method further comprising: generating a node access policy, the node access policy defining combined access rights for one or more containers on a node that includes a collection of compute instances, the method further comprising: storing the node access policy in a memory; storing the container access policy in the memory; and sending the container access policy to a request forwarder of a compute instance in a production environment, the request forwarder accessing the container access policy to grant the access rights for the container to the one or more cloud services.

2. The method of claim 1 , further comprising granting access permissions equal to the combined access of the one or more containers assigned to the node.

3. The method according to claim 1, further comprising: dividing the computing instance into a group of a plurality of nodes, each of the plurality of nodes having a separate access; and The method of claim 1 , comprising allocating one or more containers to nodes having sufficient access based at least in part on the node access policy.

4. The method of claim 1 , wherein node access is predetermined and container access within each node is dynamically set.

5. Testing the access requirements of the cloud system; and Detecting that a particular application failed to access the one or more cloud services based at least in part on entries in the request log; and The method of any of claims 1 to 4, further comprising: modifying permissions of the computing instance to correct the failure.

6. setting the request forwarder for each container type in the production environment to an authorized mode, the authorized mode granting access to the one or more cloud services for the one or more applications stored in the container; and receiving the one or more requests from the request forwarder; and switching the request forwarder to a restricted mode in response to the number of the one or more requests exceeding a threshold requirement, the restricted mode granting the one or more applications access to the one or more cloud services based in part on the container access policies.

7. 1. A computer system comprising: one or more memories; and one or more processors communicatively coupled to the one or more memories and configured to perform operations, the operations including: Receiving one or more requests to access one or more cloud services; storing the one or more requests in a request log; receiving one or more access rules applicable to cloud service access rights for the one or more cloud services; aggregating the one or more requests from the request log to determine access requirements for a container, the container configured to store one or more applications, the operations further comprising: generating a container access policy defining access rights for a container to the one or more cloud services, the container access policy being generated based at least in part on the aggregated one or more requests and the one or more access rules, the operations further comprising: generating a node access policy, the node access policy defining combined access rights for one or more containers on a node that includes a collection of compute instances, the operations further comprising: storing the node access policy in a memory; storing the container access policy in the memory; and sending the container access policy to a request forwarder of a compute instance in a production environment, the request forwarder accessing the container access policy to grant the access rights of the container to the one or more cloud services.

8. The one or more processors further comprise:

8. The computer system of claim 7, configured to perform operations including granting access permissions equal to the combined access of the one or more containers assigned to the node.

9. The one or more processors further comprise: and configured to perform operations including partitioning the computing instances into a group of multiple nodes, each of the multiple nodes having a separate access, the one or more processors further comprising: The computer system of claim 7 , configured to perform operations including assigning one or more containers to nodes having sufficient access based at least in part on the node access policy.

10. 8. The computer system of claim 7, wherein node access is predetermined and container access within each node is dynamically set.

11. The one or more processors further comprise: Testing the access requirements of the cloud system; and Detecting that a particular application failed to access the one or more cloud services based at least in part on entries in the request log; and and modifying permissions of the computing instance to correct the failure.

12. The one or more processors further comprising: and performing operations including setting the request forwarder for each container type in the production environment to an authorized mode, the authorized mode granting access to the one or more cloud services for the one or more applications stored in the container, the one or more processors further configured to: receiving the one or more requests from the request forwarder; and in response to the number of the one or more requests exceeding a threshold requirement, switching the request forwarder to a restricted mode, the restricted mode granting the one or more applications access to the one or more cloud services based in part on the container access policy.

13. An apparatus comprising means for carrying out the steps according to any one of claims 1 to 6.

14. A computer program comprising computer instructions which, when executed by a processor, perform the steps of the method according to any one of claims 1 to 6.

Images