Techniques for certificate service in an overlay network

Abstract

Techniques for implementing a certificate service in a reduced footprint data center are disclosed. A certificate service data plane can be implemented on a computing device of a reduced footprint data center. The certificate service data plane can receive a certificate signing request from an agent executing on an additional computing device communicatively connected to the computing device. The certificate signing request can include identity information corresponding to an identity of the agent. The certificate service data plane can obtain identity verification information and verify the identity of the agent by at least comparing the identity information to the identity verification information. The certificate service data plane can then issue a signed certificate to the agent.

Description

[0001] Cross-reference to related applications

[0002] This international application claims priority and benefits from the following applications, the entire contents of which are hereby incorporated herein by reference for all purposes:

[0003] 1. U.S. Provisional Patent Application No. 63 / 564,195, filed on March 12, 2024, entitled “SCALABLE FOOTPRINT FOR DEDICATED CLOUDTECHNIQUES”;

[0004] 2. U.S. Provisional Patent Application No. 63 / 568,061, filed on March 21, 2024, entitled “NETWORKING FOR A SCALABLE DEDICATED CLOUD FOOTPRINT”;

[0005] 3. U.S. Provisional Patent Application No. 63 / 568,234, filed on March 21, 2024, entitled “SCALABLE FOOTPRINT FOR DEDICATED CLOUDTECHNIQUES”;

[0006] 4. U.S. Provisional Patent Application No. 63 / 633,966, filed on April 15, 2024, entitled “SCALABLE FOOTPRINT FOR DEDICATED CLOUDTECHNIQUES”;

[0007] 5. U.S. Provisional Patent Application No. 63 / 637,691, filed on April 23, 2024, entitled “SCALABLE FOOTPRINT FOR DEDICATED CLOUDTECHNIQUES”;

[0008] 6. U.S. Provisional Patent Application No. 63 / 660,377, filed on June 14, 2024, entitled “DRCC ARCHITECTURE UPDATE”;

[0009] 7. U.S. Provisional Patent Application No. 63 / 681,059, filed August 8, 2024, entitled “TECHNIQUES FOR A CERTIFICATES SERVICE INAN OVERLAY NETWORK”; and

[0010] 8. U.S. non-provisional patent application No. 19 / 075,726, filed on March 10, 2025, entitled “TECHNIQUES FOR A CERTIFICATES SERVICE INAN OVERLAY NETWORK”. Technical Field

[0011] This disclosure generally relates to data centers. More specifically, this disclosure relates to data centers in which certificate services are implemented within an overlay network. Background Technology

[0012] Cloud service providers (CSPs) can supply computing infrastructure to customers using resources across several data centers. As cloud computing demand increases, CSPs can improve the availability of cloud resources by scaling data centers. However, scaling can result in a large data center footprint and a large number of computing devices, requiring a proportional number of resources to operate and reserving significant computing resources for the effective management of the cloud resources themselves. Summary of the Invention

[0013] Embodiments of this disclosure relate to cloud computing networks. More specifically, this disclosure describes an architecture, infrastructure, and related technologies for implementing block storage services in a reduced-footprint data center. A typical CSP can provide cloud services to multiple customers. Each customer may have the ability to customize and configure the supplied infrastructure to support the cloud resources allocated to them. To manage the infrastructure provisioning for multiple customers, the CSP may reserve computing resources within the data center to provide certain “core” services to both customers and other services operated by the CSP. For example, services such as block storage, object storage, and identity and access management are implemented within a “service enclave” of the data center. The service enclave may be connected via an underlying network of computing devices (virtual machines and / or bare metal instances) hosted within the data center. The underlying network may be part of the data center’s “underlying network,” which includes a physical network connecting the bare metal devices and SmartNICs for the computing devices. In contrast, CSP customers are supplied infrastructure in an “overlay network,” which includes one or more VCNs of a virtualized environment to provide resources (e.g., compute, storage, etc.) to customers.

[0014] Service enclaves exist on dedicated hardware within a data center. Therefore, services hosted within a service enclave are difficult to scale. While additional racks and servers can be implemented within the data center to expand resources available to CSP customers, the dedicated computing resources used for a service enclave are typically of a fixed size, dependent on the maximum projected size of the data center. Expanding a service enclave can require complex additions of computing resources, which can impact the availability of core services to customers. Additionally, unused resources within a service enclave (e.g., if the service enclave is sized too large for customer demand from the data center) are not readily available to customers because service enclaves typically do not allow network access from the customer's overlay network.

[0015] Even with increasing demand for cloud services, CSPs may want to deploy data centers with the smallest possible initial physical footprint to meet this need. This smaller footprint increases the ease of deploying physical components and configuring the initial infrastructure, while still allowing the data center to scale to meet customer demands. Instead of dedicating a portion of the computing hardware to the service enclave within the reduced footprint, the "core services" hosted in the service enclave can be implemented in the overlay network. By doing so, the core services can be scaled as the data center footprint expands. The computing equipment used to construct the reduced footprint data center can be homogenized, improving the initial configuration and facilitating footprint expansion when adding additional homogenized equipment. Additionally, by eliminating the base network, flexible overlay network shapes become available for both CSP core services and customers.

[0016] One service that can be moved to the overlay network is a Public Key Infrastructure (PKI) service. A PKI service can support authorization and authentication operations for entities within the base network, including certificates for establishing TLS channels for base hosts (including bare metal hosts and agents running on other base devices such as SmartNICs and Integrated Lights Out Managers (ILOMs)). An ILOM can be a processor or processing platform integrated with bare metal hosts in a data center, providing functionality for remote management and monitoring of hosts in situations where general host functionality may be compromised (e.g., failure occurs). A PKI service can include one or more root Certificate Authorities (CAs) for signing certificate requests from hosts utilizing the PKI service. Another related service that can be moved to the overlay network is a certificate service, which can also provide CSP clients with certificates and private CAs for TLS. For example, a certificate service can provide authorization / authentication functionality for TLS / mTLS connections for other CSP services such as Load Balancer as a Service (LBaaS). PKI and certificate services can work in conjunction with identity services implemented in the overlay network, including the Identity Data Plane (IDDP) and the implemented Identity and Access Management (IAM) services.

[0017] Due to the overlapping functionality of PKI and certificate services, services in the overlay network can be merged in a reduced-footprint data center to provide authorization / authentication and certificate services for both use cases. However, moving certificate and PKI services to the overlay network can create circular dependencies with IDDP and IAM. For example, when IDDP or IAM services are first provisioned in a reduced-footprint data center, the certificate service in the overlay may not yet be deployed to provide certificate functionality for connections to IDDP or IAM (including the connection between IDDP or IAM and the certificate data plane used to handle certificate requests). Additionally, some functionality will remain in the base layer, including SmartNIC and ILOM agents, while other core services may require certificate authorization to support mTLS for connections to the base layer's access VCN.

[0018] The embodiments described herein relate to methods, systems, and computer-readable media for implementing a certificate service data plane in an overlay network of a reduced-footprint data center and providing functionality for breaking circular dependencies between certificate services and other services implemented in the reduced-footprint data center. One method may include implementing a certificate service data plane on a computing device in the reduced-footprint data center, and receiving a certificate signing request at the certificate service data plane from an agent executing on an additional computing device communicatively connected to the computing device. The certificate signing request may include identity information corresponding to the agent's identity. The method may further include the certificate service data plane obtaining authentication information. The certificate service data plane may verify the agent's identity by at least comparing the identity information with the authentication information. The certificate service data plane may issue a signed certificate to the agent.

[0019] Another embodiment relates to a computing system including one or more processors and one or more memories storing computer-executable instructions that, when executed by the one or more processors, cause the computing system to perform the methods described above.

[0020] Another embodiment relates to a non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors of a computer system, cause the computer system to perform the methods described above. Additionally, embodiments may be implemented using a computer program product comprising a computer program / instructions that, when executed by a processor, cause the processor to perform any of the methods described in this disclosure.

[0021] Additional aspects of this disclosure include those provided in the following examples: Example 1: A method comprising implementing a certificate service data plane on a computing device in a reduced-footprint data center, the certificate service data plane including a security certificate that can be used to establish a secure communication channel; establishing a secure communication channel with a service executing in a virtual cloud network in the reduced-footprint data center, the service including a corresponding security certificate of the security certificate; receiving a certificate signing request from the service at the certificate service data plane using the secure communication channel; and issuing a signed certificate to the service by the certificate service data plane using the secure communication channel.

[0022] Example 1.1: The method of Example 1, wherein the certificate service is initially supplied with the security certificate in a data center with a reduced footprint.

[0023] Example 1.2: The method of Example 1, wherein the service is initially supplied with the corresponding security certificate in a data center with a reduced footprint.

[0024] Example 1.3: The method of Example 1, wherein the secure communication channel includes a bidirectional transport layer secure channel.

[0025] Example 1.4: The method of Example 1, wherein establishing a secure communication channel includes verifying the corresponding security certificate using the security certificate.

[0026] Example 1.5: The method of Example 1, where a secure communication channel is established to authenticate the service for a certificate signing request.

[0027] Example 1.6: The method of Example 1, wherein the security certificate is initially supplied on a secure element communicatively connected to the computing device.

[0028] Example 2: A computing system includes one or more processors and one or more memories storing computer-executable instructions that, when executed by the one or more processors, cause the computing system to perform any of the methods described in Examples 1-1.6 above.

[0029] Example 3: A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a computing system, cause the computing system to perform any of the methods of Examples 1-1.6 above. Additionally, embodiments may be implemented using a computer program product comprising a computer program / instructions that, when executed by a processor, cause the processor to perform the methods of Examples 1-1.6. Attached Figure Description

[0030] Figure 1 This is a block diagram illustrating an example system architecture for a data center with a reduced footprint, including initialization equipment, according to some embodiments.

[0031] Figure 2A This is a block diagram illustrating a typical data center that includes multiple server racks reserved for specific functions.

[0032] Figure 2B This is a block diagram illustrating a data center with a reduced footprint that provides services within an overlay network, according to some embodiments.

[0033] Figure 3 This is a block diagram illustrating a data center expansion with a reduced footprint according to some embodiments.

[0034] Figure 4 This is a block diagram illustrating the networking connection between the overlay network and the underlying network in a data center with a reduced footprint, according to some embodiments.

[0035] Figure 5This is a block diagram illustrating an example architecture of an overlay network providing certificate services in a space-saving data center, according to some embodiments.

[0036] Figure 6 This is a block diagram illustrating an example architecture of a certificate service according to some embodiments.

[0037] Figure 7 This is a flowchart of an example process for implementing certificate services in an overlay network of a data center with a reduced footprint, according to some embodiments.

[0038] Figure 8 This is a flowchart of an example process for initializing certificate services in a data center with a reduced footprint, according to some embodiments.

[0039] Figure 9 This is a block diagram illustrating a pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0040] Figure 10 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0041] Figure 11 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0042] Figure 12 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0043] Figure 13 This is a block diagram illustrating an example computer system according to at least one embodiment. Detailed Implementation

[0044] The adoption of cloud services has increased rapidly in recent years. Various types of cloud services are now offered by a wide variety of cloud service providers (CSPs). The term cloud service is generally used to refer to services or functions that a CSP provides to users or customers on demand (e.g., via a subscription model) using systems and infrastructure (cloud infrastructure) provided by the CSP. Typically, the servers and systems that constitute the CSP infrastructure and are used to provide cloud services to customers are separate from the customer's own on-premises servers and systems. Therefore, customers can utilize cloud services provided by the CSP without having to purchase separate hardware and software resources for these services. Cloud services are designed to provide subscribers with easy, scalable, and on-demand access to applications and computing resources without requiring customers to invest in the infrastructure used to provide these services or functions. Various types or models of cloud services can be provided, such as Software as a Service (SaaS), Platform as a Service (PaaS), Infrastructure as a Service (IaaS), etc. Customers can subscribe to one or more cloud services provided by a CSP. Customers can be any entity, such as individuals, organizations, enterprises, etc.

[0045] As indicated above, the CSP is responsible for providing the infrastructure and resources used to deliver cloud services to subscribers. Resources provided by the CSP can include both hardware and software resources. These resources can include, for example, computing resources (e.g., virtual machines, containers, applications, processors), storage resources (e.g., databases, data repositories), networking resources (e.g., routers, host machines, load balancers), identity, and other resources. In some implementations, the resources provided by the CSP for providing a set of cloud services are organized into a data center. A data center can be configured to provide a specific set of cloud services. The CSP is responsible for equipping the data center with the infrastructure and resources required to provide that specific set of cloud services. The CSP can build one or more data centers.

[0046] The following definitions are useful for parts of a data center built by a CSP: The underlying network is the physical network located beneath the overlay network and its Virtual Cloud Network (VCN). The existing base network is part of the underlying network. ILOM ports, management, and SmartNIC base addresses are also part of the underlying network.

[0047] Overlay network – A network environment that can be used by services and applications (including virtualized environments) that provide data center functionality to both customers and CSPs. An overlay network can include one or more VCNs, virtualized environments, and networking connections from these VCNs in a reduced-footprint data center to other cloud computing services provided by the CSP (e.g., services provided in other data center environments). The following is about… Figures 9-13Provides specific details about network virtualization and VCN as part of Infrastructure as a Service.

[0048] The base network is part of the underlying network that contains host devices (e.g., bare-metal computing devices and / or VMs) that run only base services. In existing environments, these host devices may not have SmartNICs. The host devices may be managed by a service team responsible for one or more base services within the base services.

[0049] Baseline services – A list of services currently running in the baseline network. While most of these services operate within service enclaves (block storage, object storage, identity services, etc.), some baseline services reside outside of service enclaves. Currently, baseline services are a mix of services that must communicate with the underlying network (e.g., network monitoring) and services that reside in service enclaves due to historical reasons (e.g., object storage). With the elimination of dedicated baseline hosts, we expect baseline services to converge to services that only need to communicate with the underlying network.

[0050] SmartNIC – A computing component that combines a network interface card with additional features for network virtualization to create a network abstraction layer that can run on top of physical networking components, such as the underlying network. A SmartNIC may include a processor and memory capable of performing computational operations to provide additional functionality.

[0051] Integrated Unattended Manager (ILOM) - An ILOM can be a processor or processing platform integrated with a bare-metal host in a data center, providing the ability to remotely manage and monitor the host in situations where the host's general functionality may be impaired (e.g., a failure occurs).

[0052] One or more BIOS devices – One or more computing devices on a server rack in a space-saving data center. One or more BIOS devices can be designed to enable independent and resilient operation during various boot scenarios and network outages. One or more BIOS devices can be configured to facilitate initial boot processing in a space-saving data center, provide essential services during recovery, and ensure zone stability, especially in power-constrained environments. The BIOS device hosts a range of functions that allow for autonomous operation of the zone. These functions may include, for example, DNS resolution, NTP synchronization, DHCP / ZTP configuration, and various security and provisioning services. By providing these capabilities, the BIOS device ensures that the rack can bootstrap, recover from power or network-related events, and maintain essential connectivity and management functions without relying on external resources. For example, each server rack may have one BIOS device, or may have two or three BIOS devices. In various embodiments, the BIOS device may have similar hardware specifications (e.g., number of processors, amount of memory, number of attached storage devices) to other server devices on the rack.

[0053] Reduced footprint data centers can have a new regional architecture where the initial network footprint is as small as possible (e.g., six racks, four racks, or even just one server rack) while still providing core cloud services and scalability to meet customer needs. Specifically, reduced footprint data centers may not isolate resources used for service enclaves (SEs) from resources in customer enclaves (CEs). Instead, a Butterfly region places SE services (e.g., block storage, object storage, identity) that primarily operate in the base network into the overlay network. This means that reduced footprint data centers may not have dedicated hosts for the base network and may require specific solutions for connectivity with the base services now in the overlay. Additionally, a small subset of basic bootstrapping services is needed to ensure initial routing configuration for services in the overlay during startup and / or recovery. Since the "core services" now operate in the overlay network, services such as block storage may have circular dependencies on other services such as compute. For example, computing uses block storage to provide boot volumes for VMs operating on the block storage, and the block storage uses a device encryption key service for its VMs to use computing to decrypt the boot volumes.

[0054] Figures 1-4 An overview of the concept embodied by data centers with reduced footprint is provided.

[0055] Figure 1This is a block diagram illustrating an example system architecture for a space-reduced data center 100, including initialization device 102. (See diagram for example.) Figure 1 As shown, the reduced-footprint data center 100 may include six server racks. These racks may be referred to as "butterfly" racks. The reduced-footprint data center 100 may include butterfly racks 110, 120, 130, 140, 150, and 160. In some embodiments, these racks may be identical. For example, butterfly rack 110 may include the same number of computing and / or networking devices as each of the other butterfly racks 120-160.

[0056] The Butterfly rack 110 may include two top-of-rack (TOR) switches 106 and 108. Each of the TOR switches 106 and 108 may include one or more networking switches configured to provide network communication between server devices and other computing devices within the Butterfly rack 110, and to provide one or more networking connections to other Butterfly racks 120-160 and / or other networks including the customer network 114.

[0057] The Butterfly rack 110 may also include one or more BIOS devices 102. The BIOS device may be a server device configured to perform one or more processes to provide a set of "core services" within the reduced-footprint data center 100 during boot / startup processing. The one or more BIOS devices 102 may configure one or more components of the reduced-footprint data center 100 during boot. For example, the one or more BIOS devices 102 may send network configuration information to networking devices within the Butterfly racks 110-160. The networking device may be a SmartNIC attached to a server device within the Butterfly racks 110-160. As another example, the one or more BIOS devices 102 may send network configuration information to a base access VCN. The base access VCN may be deployed to one or more hosts within the reduced-footprint data center 100. For example, a VM running in the Butterfly racks 110-160 may be configured as a base access VCN. A base access VCN can be configured to provide networking routing between networked devices (e.g., SmartNICs) and other networking components that are currently performing base services in their own VCNs within the coverage area and one or more other VCNs (e.g., customer VCNs).

[0058] In some embodiments, one or more BIOS devices 102 may also be configured to host one or more services, such as a Key Exchange Service (KeS), a Device Encryption Key (DEK) service, or other core services. Additionally, one or more BIOS devices 102 may include boot volumes for booting VMs on host devices in a reduced-footprint data center 100. For example, one or more BIOS devices 102 may provide boot volumes for VMs on hypervisors hosted on one or more server devices 104.

[0059] The butterfly rack 110 may include one or more additional server devices 104. Each of these server devices 104 may each include one or more processors and one or more memories, which together can store and execute instructions for implementing computing services as described herein, including, for example, computing, storage, VMs, CSP services, customer service, and / or applications. Figure 1 As depicted herein, each of the butterfly racks 110-160 may include the same number of server devices(s) and TORs. In some embodiments, each of the butterfly racks 110-160 may include a BIOS device, although the techniques described herein can be implemented using only a single BIOS device within a data center 100 with a reduced footprint. Each of the server devices(s) 104 may include a Trusted Platform Module (TPM). The TPM on each device may be a microcontroller or other processor(s) and a storage device for performing cryptographic operations such as hashing, encryption / decryption, key and key pair generation, and key storage. TPMs may typically conform to standards characterizing such devices, such as ISO / IEC 11889.

[0060] The reduced-footprint data center 100 may also include a networking rack 112. The networking rack 112 may include one or more networking devices, including switches, gateways, routers, etc., for communicatively coupling the butterfly racks 110-160 to each other and to a customer network 114. The customer network 114 may include an on-premises network connected to the reduced-footprint data center 100. In some embodiments, the customer network 114 may provide network connectivity to public networks, including the Internet. As described below with respect to Figure 2, the networking rack 112 may not be part of the initial “small” reduced-footprint data center and may be added to support scaling of the reduced-footprint data center 100.

[0061] Figure 2AThis is a block diagram illustrating a conventional data center 200 that includes multiple server racks reserved for specific functions. In the conventional data center 200, each of the multiple server racks may include multiple server devices, as well as networking equipment (e.g., TOR) and power and distribution equipment. Figure 2A The conventional data center 200 shown can have a standard footprint of 13 server racks as shown, although larger data centers may have additional server racks.

[0062] To provide network isolation between customer data and CSP data for CSP services executed within a conventional data center 200, a portion of the server racks can be reserved as a service enclave. This allows computing devices on those server racks to be hosted and provide CSP services within the conventional data center 200, without simultaneously hosting customer data. For example... Figure 2A As shown, server racks 1-4 can be included as service enclave racks 202.

[0063] Similarly, a portion of a server rack can be provided as a customer enclave, allowing the computing devices on those server racks to host customer services, applications, and associated customer data. Racks 5-7 can be part of customer enclave rack 204 within a conventional data center 200.

[0064] Isolation between service enclaves and customer enclaves can be enforced through software-defined perimeters, which define the edge devices and / or software within the enclave to distinguish them from hardware / software elements outside the enclave. Access to and from each enclave can be controlled, monitored, and / or policy-driven. For example, access to a service enclave can be authorization-based, restricting access to only authorized CSP clients. Such access can be based on one or more credentials provided to the enclave.

[0065] The conventional data center 200 may also include database racks 206 (racks 8-9) and networking racks 208 (racks 10-13). Database racks 206 may include computing and storage devices within the conventional data center 200 that provide storage and management for databases, data repositories, object storage, and similar data persistence technologies. Networking racks 208 may include networking devices that provide connectivity to computing devices within the conventional data center 200 and to other networks (e.g., customer networks, the Internet, etc.).

[0066] Figure 2B This is a block diagram illustrating a reduced-footprint data center 210 providing services within an overlay network, according to some embodiments. The reduced-footprint data center 210 may be... Figure 1An example of a reduced-footprint data center 100 includes six butterfly racks, each with multiple server devices, networking devices, and power distribution equipment.

[0067] Unlike a conventional data center 200 where specific server racks are reserved as service enclave racks 202 and customer enclave racks 204, a reduced-footprint data center 210 can have an overlay network 212 spanning computing devices across all server racks. For example, server devices on Butterfly Rack 1 and Butterfly Rack 6 can host VMs for VCNs within the overlay network 212. The overlay network 212 can then include both core services 214 and customer services 216. Core services 214 can include one or more VCNs for CSP services hosted within the service enclaves of the conventional data center 200 (e.g., on service enclave rack 202). In the reduced-footprint data center 210, core services 214 can reside on overlay network 212 across any one or more server devices within a Butterfly Rack. Similarly, customer services 216 can reside on overlay network 212 across host devices on any Butterfly Rack. In some embodiments, core services 214 can be hosted on specific devices within the reduced-footprint data center. For example, core service 214 can be hosted on butterfly racks 1-3, while customer service 216 can be hosted on butterfly racks 4-6. In other embodiments, core service 214 and customer service 216 can be hosted on any butterfly rack, such as... Figure 2B As depicted in the text.

[0068] Figure 3 This is an extended block diagram illustrating a reduced-footprint data center 300 according to some embodiments. The reduced-footprint data center 300 may include a plurality of reduced-footprint server racks 302. Each of the plurality of reduced-footprint server racks 302 may be as described above. Figure 1 An example of a butterfly rack from the described butterfly racks 110-160.

[0069] Multiple footprint-reduced server racks 302 can be connected in a ring network 304 using directional network connections between each seat of a TOR switch on each server rack. For example, a first TOR switch at each rack can be connected to the first TOR switches of two adjacent server racks, allowing data communication from the server racks to flow in one direction. A second TOR switch at each rack can be connected to the second TOR switches of two adjacent server racks, providing data communication between racks in opposite directions. The first and second TOR switches at each rack can be interconnected and connected to each server device on the rack, providing multiple redundant network paths from any server device in any server rack to another server device in another server rack. Therefore, the ring network 304 can allow low-latency and highly available network connectivity between resources hosted on any computing device (e.g., server device) in the multiple footprint-reduced server racks 302.

[0070] To scale the reduced footprint data center 300 from the initial footprint provided by the reduced footprint server rack 302, additional server racks can be connected to the reduced footprint server rack 302. A networking rack 306 can be implemented at the reduced footprint data center 300. The networking rack 306 can be as described above regarding... Figure 1 An example of the described networking rack 112. Networking rack 306 may include multiple networking ports that can be used to connect to one or more of a plurality of reduced-footprint server racks. For example, networking rack 306 may be connected to a first reduced-footprint server rack using connection 308. Networking rack 306 may be, for example, a dual-chassis system with four LCs, each LC having 34 x 400G ports for a total of 384 x 100G links in each chassis.

[0071] After the networking rack 306 has been implemented and connected, additional server racks 310 can be installed in the reduced-footprint data center 300. The additional server racks 310 may differ from the reduced-footprint server racks in the multiple reduced-footprint server racks 302. For example, the additional server racks 310 may include a different number of server devices, each including a different number of compute and / or storage resources (e.g., processors, processing cores, dynamic memory, non-volatile memory, etc.). Once the additional server racks 310 have been connected to the networking rack 306, cloud services hosted on the multiple reduced-footprint server racks 302 can be scaled to utilize the compute resources of the additional server racks 310. As an example, a cloud service (e.g., compute) hosted on the multiple reduced-footprint server racks 302 can provision a portion of its data plane on one of the server devices in the additional server racks 310, thereby allowing compute services to instantiate VMs on the additional server racks 310.

[0072] Figure 4 This is a block diagram illustrating an example network architecture for networking connections between one or more VCNs (baseline service VCNs) in an overlay network 402 and an underlying network 422 in a reduced-footprint data center 400 according to some embodiments. The reduced-footprint data center 400 may be an example of other reduced-footprint data centers described herein, including... Figure 1 A data center with a reduced footprint of 100.

[0073] In the reduced footprint data center 400, CSP services previously implemented in the SE (e.g., hosted on service enclave rack 202 of Figure 2) can now be executed in one or more base service VCNs 404-408. For example, base service VCN-1 404 could be a VCN for the compute service control plane, base service VCN-2 406 could be a VCN for PKI services, and base service VCN-N 408 could be a VCN for block storage services. The SE service control and data planes can be separated into different VCNs. Base service VCNs 404-408 can reside in overlay network 402. Overlay network 402 may also include one or more client VCNs 416, whose connectivity to the underlying network 422 may be limited.

[0074] Each base service VCN can have its own routing table, which defines network traffic routing rules for forwarding network traffic within the reduced footprint of the data center's 400 network. For example... Figure 4As shown, base service VCN-1 can have VCN-1 routing table 410, base service VCN-2 can have VCN-2 routing table 412, and base service VCN-N 408 can have VCN-N routing table 414. The routing information for each base service VCN in base service VCNs 404-408 can be initially configured when the reduced footprint data center 400 is first built, so that network traffic to / from the core SE service can be routed between the overlay network 402 and the underlying network 422.

[0075] The underlying network 422 may include various devices and other network endpoints connected via physical networking components of the space-reduced data center 400. For example... Figure 4 As shown, the underlying network 422 may include, but is not limited to, one or more ILOMs 424, bastion hosts 426, one or more NTP servers 428, BIOS services 430, and one or more VNICs 432. One or more ILOMs 424 may be computing devices and network targets that provide access to server equipment in the reduced-footprint data center 400 for in-band and out-of-band management. For example, one or more ILOMs 424 may allow remote management of associated server equipment within server racks in the reduced-footprint data center 400, separate from the networking path defined for the area. A bastion host 426 may be a service that runs on server equipment in the reduced-footprint data center 400, providing network access via the underlying network 422 and having no public network address. A bastion host 426 may be combined with a bastion host service operating on the underlying network 422 to provide remote access to computing resources within the reduced-footprint data center 400. A bastion host service may be an SE service of the overlay network 402 that has not been moved into the reduced-footprint data center 400. Similarly, a Network Time Protocol (NTP) server 428 may operate in the underlying network 422 to provide accurate timing for devices and services within the reduced footprint data center 400. A BIOS service 430 may include services on one or more initialization devices hosted on server racks within the reduced footprint data center 400. For example, a BIOS service 430 may include a key encryption service that can be used to encrypt / decrypt data on server devices in the reduced footprint data center 400 during initial boot processing. As another example, a BIOS service 430 may include a network configuration service that can provide initial network configuration for devices within the reduced footprint data center 400. One or more VNICs 432 may include network interfaces defined by SmartNICs connected to server devices within the reduced footprint data center 400.

[0076] As the SE service is moved to overlay network 402, it may still require network connectivity with the underlying network 422 to function properly. To provide this connectivity, a base access VCN 418 can be implemented within a smaller footprint data center 400. The base access VCN 418 may include a dynamic routing gateway (DRG) that allows communication between base service VCNs 404-408 and the underlying network 422. The base access VCN 418 may then have a DRG routing table 420, which can define a single routing rule for reaching the underlying network 422 from the base service VCNs 404-408.

[0077] To avoid circular dependencies during the initial construction of the footprint-reducing data center 400 or during recovery from an outage event, the equipment is initialized (e.g., Figure 1 The BIOS device 102 can be used to configure network addresses and routes for one or more SmartNICs in the base access VCN 418, the dynamic routing gateway within the base access VCN 418, and / or the underlying network 422. When booting server equipment into each reduced-footprint data center 400 server rack, the base access VCN 418 can be deployed to communicatively connect one or more base service VCNs 404-408 to the underlying network 422. The initialization device can send network configuration information to the base access VCN 418 to configure the DRG routing table 420, thereby providing an initial network address (e.g., an IP address) for each endpoint of the base service VCNs 404-408 in the overlay network 402 until DHCP services and other networking services are available in their respective base service VCNs.

[0078] Additionally, the initialization device can send networking configuration information to define one or more static routes for the dynamic routing gateway as part of the DRG routing table 420. Static routes can characterize the underlying network 422 (including the SmartNICs of each server device connected to the reduced footprint data center, e.g., Figure 1 Network connection between (one or more) server devices 104) and each base service VCN 404-408.

[0079] Finally, the initialization device can send network configuration information to each SmartNIC to provide each SmartNIC with a network address (e.g., the network address of the endpoint of the SmartNIC in the underlying network 422). In response to the initialization device receiving an indication that the corresponding component has been promoted to an active state (e.g., the SmartNIC is powered on and accessible through the underlying network 422, the base access VCN 418 is deployed to one or more hosts within the reduced footprint data center 400, etc.), the configuration of each of these components in the reduced footprint data center can be completed.

[0080] Certificate services in overlay networks

[0081] One service that can be moved to the overlay network is a Public Key Infrastructure (PKI) service. The PKI service can support authorization and authentication operations for entities within the base network, including certificates for establishing TLS channels for base hosts (including bare metal hosts and agents running on other base devices such as SmartNICs and Integrated Unattended Managers (ILOMs)). An ILOM can be a processor or processing platform integrated with bare metal hosts in a data center, providing functionality for remote management and monitoring of hosts in situations where general host functionality may be compromised (e.g., failure occurs). The PKI service may include one or more root Certificate Authorities (CAs) for signing certificate requests from hosts utilizing the PKI service. Another related service that can be moved to the overlay network is a certificate service, which can also provide CSP clients with certificates and private CAs for TLS. For example, the certificate service can provide authorization / authentication functionality for TLS / mTLS connections for other CSP services such as Load Balancer as a Service (LBaaS). The PKI and certificate services can work in conjunction with identity services implemented in the overlay network, including the Identity Data Plane (IDDP) and the implemented Identity and Access Management (IAM) service.

[0082] Due to the overlapping functionality of PKI and certificate services, services in the overlay network can be merged in a reduced-footprint data center to provide authorization / authentication and certificate services for both use cases. However, moving certificate and PKI services to the overlay network can create circular dependencies with IDDP and IAM. For example, when IDDP or IAM services are first provisioned in a reduced-footprint data center, the certificate service in the overlay may not yet be deployed to provide certificate functionality for connections to IDDP or IAM (including the connection between IDDP or IAM and the certificate data plane used to handle certificate requests). Additionally, some functionality will remain in the base layer, including SmartNIC and ILOM agents, while other core services may require certificate authorization to support mTLS for connections to the base layer's access VCN.

[0083] To break the circular dependencies mentioned above and provide secure connectivity for services now within the overlay, the merged certificate service can implement ephemeral certificates that can be issued by the certificate data plane, independent of the certificate control plane. To support services remaining in the underlying network (such as SmartNICs and ILOM agents), a root CA can be added to the certificate service with ephemeral certificates enabled. In a regular certificate service, certificates are supplied via the certificate control plane and retrieved from the certificate data plane. With ephemeral certificates, entities in the overlay network (including core services) can obtain certificates directly from the certificate data plane. Agents on SmartNICs and / or ILOMs can be configured to invoke the certificate data plane to obtain certificates for the corresponding instances. The certificate data plane can verify instance identity using, for example, cryptographically protected information on a Trusted Platform Module (TPM). Since the identity service may be unavailable, accessing authentication / authorization information on the TPM allows the certificate service to verify instance identity before issuing certificates to agents on SmartNICs or ILOMs.

[0084] For core services currently operating within the overlay network, including the identity data plane, the core services can invoke the certificate data plane using mTLS. An mTLS connection can be established using the mTLS certificate initially provisioned along with the core services and certificate service when they were first deployed in the reduced-footprint data center. For example, when the reduced-footprint data center is first initialized, the startup environment can be used to provision infrastructure components and deploy software for the services. At this time, an mTLS certificate can be provisioned for these services, which can then be used for authentication / authorization based on the mTLS connection used to make certificate signing requests.

[0085] Removing services from the base offers numerous advantages. Service teams can eliminate redundant work (e.g., networking configuration for underlying and overlay connectivity), CSPs can completely eliminate some services, compute and storage capacity becomes interchangeable across all services in the overlay, and service connectivity is greatly simplified. Service teams can operate without needing to know the configuration of their services. If a host is provisioned to communicate with the base's access VCN, the service functions like any other customer service. Importantly, a reduced-footprint data center does not inherently dedicate a large portion of its compute resources to CSP services in an immutable manner from the outset. If CSP services can be scaled down to meet customer demand, the freed-up resources can be offered to customers without physical expansion. Complementarily, CSP services can also scale in the same way as customer services, as the CSP services now reside in the CE overlay network. In the specific case of certificate services, services are delivered in a uniform manner across the overlay network without the need for bare-metal resources invested in the base's PKI. This allows certificate services to be implemented on a reduced butterfly footprint.

[0086] Figure 5 This is a block diagram illustrating an example architecture of an overlay network 502 providing certificate services in a footprint-reduced data center 500 according to some embodiments. The certificate service may include a certificate control plane 508 and a certificate data plane 506. The certificate data plane 506 may be hosted in a certificate VCN 510 communicatively connected to a base access VCN 512. The certificate VCN 510 may be as described above. Figure 4 The description includes examples of base service VCNs 404-408, while base access VCN 512 can be... Figure 4 An example of a baseboard connected to VCN 418. Below is about... Figures 9-13 Provide additional details about VCNs within data centers that provide Infrastructure as a Service (IaaS).

[0087] The certificate service may include a root CA and issue certificates to requesting entities within a reduced-footprint data center. These requesting entities include clients 514, services such as KMS 518, Identity Data Plane (IDDP) 522, Secret Service 526, and LBaaS 530, as well as entities in the base network 504 such as agents 542 of (one or more) SmartNICs 540 and agents 546 of (one or more) ILOMs 544. The certificate service may issue certificates using the root CA in response to a Certificate Signing Request (CSR).

[0088] Entities residing in the base network 504 within the reduced footprint data center 500 can send a Certificate Signing Request (CSR) to Certificate DP 506. When issuing certificates to these entities, Certificate DP 506 can be configured to use a temporary certificate that does not require the initial provisioning of Certificate CP 508. Alternatively, Certificate DP 506 can issue certificates directly. To authenticate the identity of an entity (e.g., an instance corresponding to Agent 542 on (one or more) SmartNIC 540s or Agent 546 on (one or more) ILOM 544s), Certificate CP 506 can obtain the identity information (e.g., a cryptographic key stored in the TPM) of the agent (e.g., Agent 542 or Agent 546). For example, the identity information could be a cryptographic hash of the identity information corresponding to the agent. The identity information can be stored in the TPM as part of the initial configuration of the bare-metal host equipment of the reduced footprint data center 500 when the reduced footprint data center 500 is first constructed. By using TPM certificate authentication 550, certificate DP 506 can verify the identity of the requesting agent without relying on IDDP 522 or having the information provided by certificate CP 508. Certificate DP 506 can be provided with a root CA for the instance corresponding to agents 542 and 546, enabling the issued certificate to establish a chain of trust for agents 542 and 546.

[0089] like Figure 5As depicted, services operating in overlay network 502 can each be hosted in their own VCN. For example, KMS 518 can be hosted in KMS VCN 516, IDDP 522 in IDDP VCN 520, Secret Service 526 in Secret VCN 524, and LBaaS 530 in LBaaS VCN 528. After these services are provisioned in the reduced footprint data center 500, they can request certificates that can be used to authenticate with other services. However, when IDDP 522 is first provisioned in the reduced footprint data center 500, the certificate service may not be able to invoke IDDP 522 when authenticating a CSR from another service. To provide authentication, the requesting service can establish an mTLS connection with Certificate DP 506 (as shown in mTLS Certificate Authentication 552). To establish an mTLS connection, a corresponding mTLS certificate can be provisioned for Certificate DP 506 and each of the core services configured to use mTLS Certificate Authentication 552. For example, during the construction of a reduced-footprint data center 500, a startup environment can be used to deploy Certificate Service DP and core service components to the reduced-footprint data center 500, including the corresponding mTLS certificate for each core service. The corresponding mTLS certificate can be used to establish a trusted and secure mTLS channel. Based on a successful mTLS connection, Certificate DP 506 can verify the identity of the requesting service and issue the requested certificate.

[0090] Figure 6 This is a block diagram illustrating an example architecture of a certificate service 600 according to some embodiments. The certificate service 600 may include a certificate CP 604 (e.g., Figure 5 Certificate CP 508) and Certificate DP 604 (e.g., Figure 5The architecture of Certificate Service 600 describes the process of implementing temporary certificate signing by Certificate DP 604. For example, CA administrator 608 can instruct Certificate CP 602 to enable temporary certificate signing. Certificate CP 602 can then use the communication path of Certificate DP 604 to enable temporary certificate signing by Certificate DP 604, instead of Certificate CP 602 providing the certificate at Certificate DP 604 itself. The configuration of Certificate DP 604 can expose APIs for Certificate DP 604 to process CSRs. After temporary certificate signing has been enabled at Certificate DP 604, a CSR from the corresponding CA user 608 can be sent to Certificate DP 604. In response to the CSR, Certificate DP 604 can authenticate, sign, and issue a signed certificate. By allowing Certificate DP 604 to directly support CSR signing, PKI agents in the underlying network and / or base network (e.g., base network 504) (e.g., Figure 5 (Proxy 542, 546). Certificate DP 604 handling of CSR signatures also allows Certificate Service 600 to have high throughput and availability in data centers with reduced footprint.

[0091] Figure 7 This is a flowchart of an example process 700 for implementing certificate services in a data center with a reduced footprint, according to some embodiments. The certificate service may include the above-mentioned... Figure 5 Examples of certificate services described are for certificates CP 508 and DP 506. Processing 700 operations can be performed by one or more computing devices (including...) in a data center with a reduced footprint. Figure 1 The computing device(s) 104) are used to perform the operation, which hosts certificate services and / or other services operating in a reduced-footprint data center. In some embodiments, a computer-readable medium includes computer-readable instructions that, when executed by one or more processors of the distributed computing system, cause the distributed computing system to perform processing 700. The operations of processing 700 can be performed in any suitable order, and processing 700 can include more than Figure 7 The operations described in the text are more or fewer operations.

[0092] Processing 700 (or any other processing and / or method described herein, including processing 800, or variations and / or combinations thereof) in whole or in part can be executed under the control of one or more computer systems configured with executable instructions, and can be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that executes jointly on one or more processors, implemented by hardware or a combination thereof. The code can be stored, for example, on a computer-readable storage medium in the form of a computer program comprising multiple instructions executable by one or more processors. The computer-readable storage medium can be non-transitory.

[0093] Processing 700 can begin at box 702 by implementing a certificate service data plane (e.g., certificate DP 506) on a computing device in a data center with a reduced footprint. The certificate service data plane can be hosted within a service VCN (e.g., certificate VCN 510) in an overlay network (e.g., overlay network 502).

[0094] At box 704, the certificate service data plane can receive certificate signing requests from an agent executing on an attached computing device communicatively connected to the computing device. For example, the attached computing device could be an agent of a managed instance (e.g., Figure 5 SmartNIC (e.g., agent 542) Figure 5 (SmartNIC 540). In some embodiments, the SmartNIC may connect to a server device in a reduced-footprint data center that hosts the certificate service data plane. In some embodiments, the additional computing device may include an ILOM, which may be the processor or processing platform of the computing device, separate from the processor used to perform the computing tasks of the computing device. The certificate signing request may include identity information corresponding to the identity of the agent. For example, the certificate signing request may include an instance identifier for the instance corresponding to the agent. In some examples, the identity information may be used to generate a cryptographic hash or other cryptographic information using one or more cryptographic hash functions.

[0095] At box 706, the Certificate Service Data Plane (TPM) can obtain authentication information. For example, the authentication information may include password information corresponding to an instance identity. In some embodiments, obtaining authentication information may include obtaining authentication information from a Trusted Platform Module (TPM) connected to the computing device. For example, the TPM may store a password hash or password key corresponding to the agent's identity information. In some embodiments, the authentication information may include a root certificate corresponding to a root certificate authority.

[0096] At box 708, the Certificate Services Data Plane can verify the identity of a proxy by at least comparing the identity information with the authentication information. For example, the Certificate Services Data Plane can generate a hash using the identity information and compare that hash with the authentication information from the TPM. If the identity information corresponds to the authentication information and the proxy instance is the identity it presents in the CSR, then the hash will match.

[0097] At box 710, the Certificate Service Data Plane can issue signed certificates to agents. The signed certificate may include a signature generated by the Certificate Service Data Plane and included in a certificate presented as part of a CSR received by the Certificate Service Data Plane. The signed certificate may be issued in part based on successful verification of the identity of the agent sending the CSR.

[0098] Figure 8 This is a flowchart of an example process 800 for initializing a certificate service in a space-reduced data center, according to some embodiments. The certificate service may be an example including certificate CP 208 and certificate DP 206 described above with respect to FIG2. The operation of process 800 may be performed by one or more computing devices (including...) in a space-reduced data center. Figure 1 The one or more server devices 104) are used to perform the certificate service and / or other services that operate in a reduced-footprint data center.

[0099] Processing 800 can begin at box 802 by implementing a certificate service data plane on computing equipment in a data center with a reduced footprint. The certificate service data plane can include security certificates that can be used to establish secure communication channels. For example, one or more mTLS certificates can be initially supplied to the certificate service data plane, which can be used to establish mTLS connections to services holding the corresponding mTLS certificates.

[0100] At box 804, a service performing in a virtual cloud network within a reduced-footprint data center can establish a secure communication channel with the certificate service data plane. This service may include a corresponding security certificate for the security certificate. In some embodiments, the secure communication channel may include a bidirectional transport layer secure channel. Establishing a secure communication channel may rely on the fact that the security certificate and the corresponding security certificate are verified / authenticated by both the certificate service and the service itself. In doing so, the establishment of the secure communication channel provides the certificate service with authentication of the service.

[0101] At box 806, the Certificate Services data plane can receive certificate signing requests from the service using a secure communication channel. Because the secure communication channel provides authentication for the service, the Certificate Services data plane can sign the certificate information contained in the certificate signing request without performing additional verification operations on the CSR (since the identity service that would allow such verification may not yet be operating in a reduced-footprint data center).

[0102] At box 808, the Certificate Service data plane can issue signed certificates to the service using a secure communication channel. The signed certificate can be sent to the requesting service via the secure communication channel.

[0103] Example Infrastructure as a Service architecture

[0104] As noted above, Infrastructure as a Service (IaaS) is a specific type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, cloud providers can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, IaaS providers can also provision various services to accompany these infrastructure components (example services include billing software, monitoring software, logging software, load balancing software, and clustering software, etc.). Therefore, because these services may be policy-driven, IaaS users can implement policies to drive load balancing to maintain application availability and performance.

[0105] In some cases, IaaS customers can access resources and services over a wide area network (WAN) such as the internet and can use the cloud provider's services to install the remaining elements of their application stack. For example, a user can log in to the IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create buckets for workloads and backups, and even install enterprise software into that VM. The customer can then use the provider's services to perform various functions, including balancing network traffic, troubleshooting application issues, monitoring performance, and managing disaster recovery.

[0106] In most cases, cloud computing models may require the involvement of cloud providers. Cloud providers can, but are not necessarily, third-party providers specializing in (e.g., provisioning, renting, selling) IaaS services. Entities may also choose to deploy private clouds, thus becoming their own infrastructure service providers.

[0107] In some examples, IaaS deployment is the process of placing a new application or a new version of an application onto a prepared application server, etc. It may also include the processing of server preparation (e.g., installation libraries, daemons, etc.). This is typically managed by the cloud provider, below the hypervisor layer (e.g., servers, storage devices, network hardware, and virtualization). Therefore, the customer can be responsible for processing (OS), middleware, and / or application deployment (e.g., on self-service virtual machines, etc., which can be started on demand).

[0108] In some examples, IaaS provisioning can refer to acquiring computers or virtual hosts for use, or even installing necessary libraries or services on them. In most cases, deployment does not include provisioning, and provisioning may need to be performed first.

[0109] In some cases, IaaS provisioning presents two distinct challenges. First, there are initial challenges in provisioning an initial set of infrastructure before anything is operational. Second, once everything is provisioned, there are challenges in evolving the existing infrastructure (e.g., adding new services, changing services, removing services, etc.). In some cases, both challenges can be addressed by enabling configuration that declaratively defines the infrastructure. In other words, the infrastructure (e.g., which components are needed and how they interact) can be defined by one or more configuration files. Therefore, the overall topology of the infrastructure (e.g., which resources depend on which resources and how they work together) can be described declaratively. In some cases, once the topology is defined, workflows for creating and / or managing the different components described in the configuration files can be generated.

[0110] In some examples, the infrastructure can have many interconnected components. For example, there may be one or more Virtual Private Clouds (VPCs) (e.g., potential on-demand pools of configurable and / or shared computing resources), also known as the core network. In some examples, one or more inbound / outbound traffic group rules may also be provided to define how inbound / outbound traffic to the network and one or more virtual machines (VMs) will be configured. Other infrastructure elements, such as load balancers, databases, etc., may also be provided. The infrastructure can evolve incrementally as more and / or more infrastructure elements are expected and added.

[0111] In some cases, continuous deployment techniques can be employed to enable the deployment of infrastructure code across various virtual computing environments. Furthermore, the described techniques enable infrastructure management within these environments. In some examples, service teams may write code that they intend to deploy to one or more, but often many, different production environments (e.g., across various geographical locations, sometimes spanning the entire world). However, in some examples, the infrastructure on which the code will be deployed may be established first. In some cases, provisioning can be done manually, resources can be provisioned using provisioning tools, and / or once the infrastructure is provisioned, the code can be deployed using deployment tools.

[0112] Figure 9 This is a block diagram 900 illustrating an example pattern of an IaaS architecture according to at least one embodiment. Service operator 902 may communicatively couple to a secure host lease 904, which may include a virtual cloud network (VCN) 906 and a secure host subnet 908. In some examples, service operator 902 may use one or more client computing devices, which may be portable handheld devices (e.g., iPhone®, cellular phone, iPad®, computing tablet, personal digital assistant (PDA)) or wearable devices (e.g., Google Glass® head-mounted display), running software (such as Microsoft Windows Mobile®) and / or various mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 8, Palm OS, etc.), and enabled for the Internet, email, short message service (SMS), Blackberry®, or other communication protocols. Alternatively, client computing devices may be general-purpose personal computers, including, for example, personal computers and / or laptops running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems. The client computing device can be a workstation computer running any of the various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, such as Google Chrome OS). Alternatively or additionally, the client computing device can be any other electronic device, such as a thin client computer, an internet-enabled gaming system (e.g., a Microsoft Xbox game console with or without Kinect® gesture input), and / or a personal messaging device capable of communicating over a network that can access VCN 906 and / or the internet.

[0113] VCN 906 may include a local peering gateway (LPG) 910, which may be communicatively coupled to a secure shell (SSH) VCN 912 via LPG 910 contained in SSH VCN 912. SSH VCN 912 may include an SSH subnet 914, and SSH VCN 912 may be communicatively coupled to a control plane VCN 916 via LPG 910 contained in control plane VCN 916. Furthermore, SSH VCN 912 may be communicatively coupled to a data plane VCN 918 via LPG 910. Control plane VCN 916 and data plane VCN 918 may be contained within a service lease 919 that may be owned and / or operated by an IaaS provider.

[0114] The control plane VCN 916 may include a control plane demilitarized zone (DMZ) layer 920 that acts as a peripheral network (e.g., a portion of a corporate network between a corporate intranet and an external network). DMZ-based servers may have limited liability and help contain vulnerabilities. Additionally, the DMZ layer 920 may include one or more load balancer (LB) subnets 922, a control plane application layer 924 that may include one or more application subnets 926, and a control plane data layer 928 that may include one or more database (DB) subnets 930 (e.g., one or more front-end DB subnets and / or one or more back-end DB subnets). One or more LB subnets 922 contained in the control plane DMZ layer 920 may be communicatively coupled to one or more application subnets 926 contained in the control plane application layer 924 and an Internet gateway 934 that may be contained in the control plane VCN 916. The application subnets 926 may be communicatively coupled to one or more DB subnets 930 contained in the control plane data layer 928, as well as a service gateway 936 and a Network Address Translation (NAT) gateway 938. The control plane VCN 916 may include the service gateway 936 and the NAT gateway 938.

[0115] The control plane VCN 916 may include a data plane mirror application layer 940, which may include one or more application subnets 926. The one or more application subnets 926 included in the data plane mirror application layer 940 may include a virtual network interface controller (VNIC) 942 capable of executing a compute instance 944. The compute instance 944 may communicatively couple the one or more application subnets 926 of the data plane mirror application layer 940 to the one or more application subnets 926 that may be included in the data plane application layer 946.

[0116] Data plane VCN 918 may include data plane application layer 946, data plane DMZ layer 948, and data plane data layer 950. Data plane DMZ layer 948 may include one or more LB subnets 922 communicatively coupled to one or more application subnets 926 of data plane application layer 946 and Internet gateway 934 of data plane VCN 918. One or more application subnets 926 may be communicatively coupled to service gateway 936 and NAT gateway 938 of data plane VCN 918. Data plane data layer 950 may also include one or more DB subnets 930 communicatively coupled to one or more application subnets 926 of data plane application layer 946.

[0117] The Internet gateway 934 of the control plane VCN 916 and data plane VCN 918 can be communicatively coupled to the metadata management service 952, which can be communicatively coupled to the public Internet 954. The public Internet 954 can be communicatively coupled to the NAT gateway 938 of the control plane VCN 916 and data plane VCN 918. The service gateway 936 of the control plane VCN 916 and data plane VCN 918 can be communicatively coupled to the cloud service 956.

[0118] In some examples, the service gateway 936 of the control plane VCN 916 or data plane VCN 918 can make application programming interface (API) calls to the cloud service 956 without traversing the public internet 954. API calls from the service gateway 936 to the cloud service 956 can be unidirectional: the service gateway 936 can make API calls to the cloud service 956, and the cloud service 956 can send the requested data to the service gateway 936. However, the cloud service 956 may not initiate API calls to the service gateway 936.

[0119] In some examples, secure host lease 904 can be directly connected to service lease 919, which would otherwise be isolated. Secure host subnet 908 can communicate with SSH subnet 914 via LPG 910, which enables bidirectional communication on otherwise isolated systems. Connecting secure host subnet 908 to SSH subnet 914 allows secure host subnet 908 to access other entities within service lease 919.

[0120] Control plane VCN 916 allows users of service lease 919 to establish or otherwise provision desired resources. Desired resources provisioned in control plane VCN 916 can be deployed or otherwise used in data plane VCN 918. In some examples, control plane VCN 916 can be isolated from data plane VCN 918, and the data plane mirror application layer 940 of control plane VCN 916 can communicate with the data plane application layer 946 of data plane VCN 918 via VNIC 942, which can be included in both the data plane mirror application layer 940 and the data plane application layer 946.

[0121] In some examples, users or clients of the system can make requests, such as create, read, update, or delete (CRUD) operations, via the public internet 954, which can transmit requests to the metadata management service 952. The metadata management service 952 can transmit the request to the control plane VCN 916 via internet gateway 934. The request can be received by one or more LB subnets 922 contained in the control plane DMZ layer 920. The LB subnets 922 can determine that the request is valid, and in response to this determination, they can transmit the request to one or more application subnets 926 contained in the control plane application layer 924. If the request is validated and requires a call to the public internet 954, the call to the public internet 954 can be transmitted to a NAT gateway 938 that can make calls to the public internet 954. The request may expect the storage to be located in one or more DB subnets 930.

[0122] In some examples, the data plane mirroring application layer 940 can facilitate direct communication between the control plane VCN 916 and the data plane VCN 918. For example, it may be desirable to apply configuration changes, updates, or other appropriate modifications to resources contained in the data plane VCN 918. Through VNIC 942, the control plane VCN 916 can communicate directly with the resources contained in the data plane VCN 918, and thus can perform configuration changes, updates, or other appropriate modifications.

[0123] In some embodiments, the control plane VCN 916 and data plane VCN 918 may be contained within a service lease 919. In this case, the system's users or customers may not own or operate the control plane VCN 916 or data plane VCN 918. Alternatively, the IaaS provider may own or operate both the control plane VCN 916 and data plane VCN 918, and both planes may be contained within the service lease 919. This embodiment can enable network isolation that prevents users or customers from interacting with the resources of other users or customers. Furthermore, this embodiment can allow users or customers of the system to privately store databases without relying on the public Internet 954, which may not have the desired threat level for storage.

[0124] In other embodiments, one or more LB subnets 922 included in the control plane VCN 916 may be configured to receive signals from the service gateway 936. In this embodiment, the control plane VCN 916 and the data plane VCN 918 may be configured to be invoked by the IaaS provider's customers without invoking the public internet 954. The IaaS provider's customers may expect this embodiment because the database(s) used by the customer can be controlled by the IaaS provider and can be stored on a service lease 919 that can be isolated from the public internet 954.

[0125] Figure 10 This is a block diagram 1000 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 1002 (e.g., Figure 9 The service provider 902 can communicatively couple to the secure host lease 1004 (e.g., Figure 9 Secure hosting lease 904), the secure hosting lease may include a Virtual Cloud Network (VCN) 1006 (e.g., Figure 9 VCN906) and Security Host Subnet 1008 (e.g., Figure 9 The secure host subnet 908). VCN 1006 may include a local peering gateway (LPG) 1010 (e.g., Figure 9 The LPG 910 can be communicatively coupled to the Secure Shell (SSH) VCN 1012 (e.g., LPG 910) contained in the SSH VCN 1012. Figure 9 SSH VCN 912). SSH VCN 1012 can include SSH subnet 1014 (e.g., Figure 9 SSH subnet 914), and SSH VCN 1012 can be communicatively coupled to control plane VCN 1016 via LPG 1010 included in control plane VCN 1016 (e.g., Figure 9 Control plane VCN 916). Control plane VCN 1016 may be included in service lease 1019 (e.g., Figure 9 In the service lease 919), and the data plane VCN 1018 (e.g., Figure 9 The data plane VCN 918 can be included in a customer lease 1021 that can be owned or operated by the system's users or customers.

[0126] The control plane VCN 1016 may include one or more LB subnets 1022 (e.g., Figure 9 The control plane DMZ layer 1020 of (one or more) LB subnets 922) (e.g., Figure 9 The control plane DMZ layer 920 may include (one or more) application subnets 1026 (e.g., Figure 9 The control plane application layer 1024 of (one or more) application subnets 926 (e.g., Figure 9 The control plane application layer 924) may include one or more database (DB) subnets 1030 (e.g., similar to...). Figure 9 The control plane data layer 1028 of (one or more) DB subnets 930 (e.g., Figure 9 The control plane data layer 928). One or more LB subnets 1022 contained in the control plane DMZ layer 1020 can be communicatively coupled to one or more application subnets 1026 contained in the control plane application layer 1024 and an Internet gateway 1034 that can be contained in the control plane VCN 1016 (e.g., Figure 9 Internet gateway 934), and application subnet(s) 1026 can communicatively couple to DB subnet(s) 1030 contained in control plane data layer 1028 and service gateway 1036 (e.g., Figure 9 Service gateway 936) and Network Address Translation (NAT) gateway 1038 (e.g., Figure 9 (NAT gateway 938). The control plane VCN 1016 may include the service gateway 1036 and the NAT gateway 1038.

[0127] The control plane VCN 1016 may include a data plane mirror of the application layer 1040, which may include one or more application subnets 1026 (e.g., Figure 9 The data plane mirror application layer 940). One or more application subnets 1026 contained in the data plane mirror application layer 1040 may include compute instances 1044 (e.g., similar to...). Figure 9The virtual network interface controller (VNIC) 1042 (e.g., the VNIC of 942) of the computing instance 944. The computing instance 1044 may facilitate the mirroring of the application subnet(s) 1026 of the application layer 1040 in the data plane and may be included in the application layer 1046 in the data plane (e.g., Figure 9 Communication between one or more application subnets 1026 in the data plane application layer 946 via VNIC 1042 contained in the data plane mirror application layer 1040 and VNIC 1042 contained in the data plane application layer 1046.

[0128] The Internet gateway 1034, included in the control plane VCN 1016, can be communicatively coupled to the metadata management service 1052 (e.g., Figure 9 Metadata management service 952), which can communicatively couple to the public Internet 1054 (e.g., Figure 9 The public internet 1054 can communicatively couple to a NAT gateway 1038 contained in a control plane VCN 1016. The service gateway 1036 contained in the control plane VCN 1016 can communicatively couple to a cloud service 1056 (e.g., ...). Figure 9 Cloud services (956).

[0129] In some examples, data plane VCN 1018 may be included in customer lease 1021. In this case, the IaaS provider may provide control plane VCN 1016 for each customer, and the IaaS provider may set up a unique compute instance 1044 for each customer, included in service lease 1019. Each compute instance 1044 may allow communication between control plane VCN 1016 included in service lease 1019 and data plane VCN 1018 included in customer lease 1021. Compute instance 1044 may allow resources provisioned in control plane VCN 1016 included in service lease 1019 to be deployed or otherwise used in data plane VCN 1018 included in customer lease 1021.

[0130] In other examples, an IaaS provider's customer may have a database residing in customer lease 1021. In this example, control plane VCN 1016 may include a data plane mirror application layer 1040, which may include one or more application subnets 1026. Data plane mirror application layer 1040 may reside in data plane VCN 1018, but it may not reside in data plane VCN 1018. That is, data plane mirror application layer 1040 may access customer lease 1021, but it may not reside in data plane VCN 1018 or be owned or operated by an IaaS provider's customer. Data plane mirror application layer 1040 may be configured to invoke data plane VCN 1018, but it may not be configured to invoke any entity contained in control plane VCN 1016. Customers may expect to deploy or otherwise use resources provided in the control plane VCN 1016 in the data plane VCN 1018, and the data plane mirroring application layer 1040 can facilitate customers' expected deployments or other uses of resources.

[0131] In some embodiments, an IaaS provider's customer can apply filters to data plane VCN 1018. In this embodiment, the customer can determine what data plane VCN 1018 can access, and the customer can restrict access from data plane VCN 1018 to the public internet 1054. The IaaS provider may not be able to apply filters or otherwise control data plane VCN 1018's access to any external networks or databases. Applying filters and controls by the customer to data plane VCN 1018, contained in customer lease 1021, can help isolate data plane VCN 1018 from other customers and the public internet 1054.

[0132] In some embodiments, cloud service 1056 may be invoked by service gateway 1036 to access services that may not exist on public internet 1054, control plane VCN 1016, or data plane VCN 1018. The connection between cloud service 1056 and control plane VCN 1016 or data plane VCN 1018 may not be real-time or continuous. Cloud service 1056 may reside on different networks owned or operated by an IaaS provider. Cloud service 1056 may be configured to receive calls from service gateway 1036 and may be configured not to receive calls from public internet 1054. Some cloud services 1056 may be isolated from other cloud services 1056, and control plane VCN 1016 may be isolated from cloud services 1056 that may not be in the same region as control plane VCN 1016. For example, control plane VCN 1016 may be located in "Region 1," and cloud service "Deployment 9" may be located in both Region 1 and "Region 2." If a call to deployment 9 is made by service gateway 1036 contained in control plane VCN 1016 located in region 1, then the call can be transmitted to deployment 9 in region 1. In this example, control plane VCN 1016 or deployment 9 in region 1 may be uncoupled from or otherwise communicate with deployment 9 in region 2.

[0133] Figure 11 This is a block diagram 1100 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 1102 (e.g., Figure 9 The service provider 902 can communicatively couple to the secure host lease 1104 (e.g., Figure 9 Secure hosting lease 904), the secure hosting lease 1104 may include a virtual cloud network (VCN) 1106 (e.g., Figure 9 VCN 906) and Secure Host Subnet 1108 (e.g., Figure 9 The secure host subnet 908). VCN 1106 may include LPG1110 (e.g., Figure 9 The LPG 910 can be communicatively coupled to the SSH VCN 1112 via the LPG 1110 contained in the SSH VCN 1112 (e.g., Figure 9 SSH VCN 912). SSH VCN 1112 can include SSH subnet 1114 (e.g., Figure 9 SSH subnet 914), and SSH VCN 1112 can be communicatively coupled to control plane VCN 1116 via LPG 1110 contained in control plane VCN 1116 (e.g., Figure 9The control plane VCN 916) and coupled to the data plane VCN 1118 via the LPG 1110 contained in the data plane VCN 1118 (e.g., Figure 9 Data plane 918). Control plane VCN 1116 and data plane VCN 1118 can be included in service lease 1119 (e.g., Figure 9 In the service rental (919).

[0134] The control plane VCN 1116 may include one or more load balancer (LB) subnets 1122 (e.g., Figure 9 The control plane DMZ layer 1120 of (one or more) LB subnets 922) (e.g., Figure 9 The control plane DMZ layer 920 may include one or more application subnets 1126 (e.g., similar to...). Figure 9 The control plane application layer 1124 of (one or more) application subnets 926 (e.g., Figure 9 The control plane application layer 924) may include a control plane data layer 1128 (e.g., DB subnet 1130) that may include one or more DB subnets 1130. Figure 9 The control plane data layer 928). One or more LB subnets 1122 contained in the control plane DMZ layer 1120 can be communicatively coupled to one or more application subnets 1126 contained in the control plane application layer 1124 and an Internet gateway 1134 that can be contained in the control plane VCN 1116 (e.g., Figure 9 Internet gateway 934), and application subnet(s) 1126 can communicatively couple to DB subnet(s) 1130 contained in control plane data layer 1128 and service gateway 1136 (e.g., Figure 9 The service gateway) and Network Address Translation (NAT) gateway 1138 (e.g., Figure 9 (NAT gateway 938). The control plane VCN 1116 may include the service gateway 1136 and the NAT gateway 1138.

[0135] Data plane VCN 1118 may include data plane application layer 1146 (e.g., Figure 9 Data plane application layer 946), data plane DMZ layer 1148 (e.g., Figure 9 Data plane DMZ layer 948), and data plane data layer 1150 (e.g., Figure 9The data plane data layer 950). The data plane DMZ layer 1148 may include one or more trusted application subnets 1160 and one or more untrusted application subnets 1162 that are communicatively coupled to the data plane application layer 1146, and one or more LB subnets 1122 of the Internet gateway 1134 contained in the data plane VCN 1118. One or more trusted application subnets 1160 may be communicatively coupled to the service gateway 1136 contained in the data plane VCN 1118, the NAT gateway 1138 contained in the data plane VCN 1118, and one or more DB subnets 1130 contained in the data plane data layer 1150. One or more untrusted application subnets 1162 may be communicatively coupled to the service gateway 1136 contained in the data plane VCN 1118 and the one or more DB subnets 1130 contained in the data plane data layer 1150. The data plane data layer 1150 may include one or more DB subnets 1130 that can be communicatively coupled to the service gateway 1136 contained in the data plane VCN 1118.

[0136] One or more untrusted application subnets 1162 may include one or more primary VNICs 1164(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 1166(1)-(N). Each tenant VM 1166(1)-(N) may be communicatively coupled to a corresponding application subnet 1167(1)-(N) that may be contained in a corresponding container egress VCN 1168(1)-(N) that may be contained in a corresponding customer lease 1170(1)-(N). A corresponding secondary VNIC 1172(1)-(N) may facilitate communication between one or more untrusted application subnets 1162 contained in data plane VCN 1118 and application subnets contained in container egress VCN 1168(1)-(N). Each container exit VCN 1168(1)-(N) may include a NAT gateway 1138, which can communicatively couple to the public Internet 1154 (e.g., Figure 9 (Public Internet 954).

[0137] Internet gateway 1134, contained in control plane VCN 1116 and data plane VCN 1118, can be communicatively coupled to metadata management service 1152 (e.g., Figure 9A metadata management system 952 is provided, which can communicatively couple to the public internet 1154. The public internet 1154 can communicatively couple to a NAT gateway 1138 contained in a control plane VCN 1116 and a data plane VCN 1118. A service gateway 1136 contained in a control plane VCN 1116 and a data plane VCN 1118 can communicatively couple to a cloud service 1156.

[0138] In some embodiments, the data plane VCN 1118 may be integrated with the customer lease 1170. Such integration may be useful or desirable for the IaaS provider's customers in certain situations (such as when support may be expected when executing code). Customers may provide code that could be destructive, might communicate with other customer resources, or might otherwise cause undesirable effects. In response, the IaaS provider may determine whether to run the code provided by the customer.

[0139] In some examples, an IaaS provider's customer may grant the IaaS provider temporary network access and request functionality attached to data plane layer application 1146. The code running this functionality may execute in VMs 1166(1)-(N) and may not be configured to run anywhere else on data plane VCN 1118. Each VM 1166(1)-(N) may be connected to a customer lease 1170. The corresponding container 1171(1)-(N) contained in VMs 1166(1)-(N) may be configured to run the code. In this case, dual isolation may exist (e.g., container 1171(1)-(N) runs the code, where container 1171(1)-(N) may be contained in at least one or more untrusted application subnets 1162 containing VMs 1166(1)-(N)), which can help prevent incorrect or otherwise unintended code from corrupting the IaaS provider's network or the networks of different customers. Containers 1171(1)-(N) may be communicatively coupled to customer lease 1170 and may be configured to transmit or receive data from customer lease 1170. Containers 1171(1)-(N) may not be configured to transmit or receive data from any other entity in data plane VCN 1118. After the code execution is complete, the IaaS provider may terminate or otherwise dispose of containers 1171(1)-(N).

[0140] In some embodiments, one or more untrusted application subnets 1160 may run code that can be owned or operated by an IaaS provider. In this embodiment, one or more trusted application subnets 1160 may be communicatively coupled to one or more database subnets 1130 and configured to perform CRUD operations in one or more database subnets 1130. One or more untrusted application subnets 1162 may be communicatively coupled to one or more database subnets 1130, but in this embodiment, one or more untrusted application subnets may be configured to perform read operations in one or more database subnets 1130. Containers 1171(1)-(N) that may be contained in each customer's VM 1166(1)-(N) and may run code from the customer may not be communicatively coupled to one or more database subnets 1130.

[0141] In other embodiments, the control plane VCN 1116 and the data plane VCN 1118 may be coupled without direct communication. In this embodiment, there may be no direct communication between the control plane VCN 1116 and the data plane VCN 1118. However, communication can occur indirectly through at least one method. The LPG 1110 may be established by an IaaS provider, which can facilitate communication between the control plane VCN 1116 and the data plane VCN 1118. In another example, either the control plane VCN 1116 or the data plane VCN 1118 may invoke the cloud service 1156 via the service gateway 1136. For example, an invocation of the cloud service 1156 from the control plane VCN 1116 may include a request for a service that can communicate with the data plane VCN 1118.

[0142] Figure 12 This is a block diagram 1200 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 1202 (e.g., Figure 9 The service provider 902 can communicatively couple to the secure host lease 1204 (e.g., Figure 9 Secure hosting lease 904), the secure hosting lease may include a Virtual Cloud Network (VCN) 1206 (e.g., Figure 9 VCN906) and Security Host Subnet 1208 (e.g., Figure 9 The secure host subnet 908). VCN 1206 can include LPG 1210 (e.g., Figure 9 The LPG 910 can be delivered via SSH VCN 1212 (e.g., LPG 910). Figure 9The LPG 1210 in SSH VCN 1212 is communicatively coupled to SSH VCN 1212. SSH VCN 1212 may include SSH subnet 1214 (e.g., Figure 9 SSH subnet 914), and SSH VCN 1212 can be communicatively coupled to control plane VCN 1216 via LPG 1210 included in control plane VCN 1216 (e.g., Figure 9 The control plane VCN 916) and coupled to the data plane VCN 1218 via the LPG 1210 contained in the data plane VCN 1218 (e.g., Figure 9 Data plane 918). Control plane VCN 1216 and data plane VCN 1218 can be included in service lease 1219 (e.g., Figure 9 In the service rental (919).

[0143] The control plane VCN 1216 may include one or more LB subnets 1222 (e.g., Figure 9 The control plane DMZ layer 1220 of (one or more) LB subnets 922) (e.g., Figure 9 The control plane DMZ layer 920 may include (one or more) application subnets 1226 (e.g., Figure 9 The control plane application layer 1224 of (one or more) application subnets 926 (e.g., Figure 9 The control plane application layer 924) may include (one or more) DB subnets 1230 (e.g., Figure 11 The control plane data layer 1228 of (one or more) DB subnets 1130 (e.g., Figure 9 The control plane data layer 928). One or more LB subnets 1222 contained in the control plane DMZ layer 1220 can be communicatively coupled to one or more application subnets 1226 contained in the control plane application layer 1224 and an Internet gateway 1234 that can be contained in the control plane VCN 1216 (e.g., Figure 9 Internet gateway 934), and application subnet(s) 1226 can communicatively couple to DB subnet(s) 1230 contained in control plane data layer 1228 and service gateway 1236 (e.g., Figure 9 The service gateway) and Network Address Translation (NAT) gateway 1238 (e.g., Figure 9 (NAT gateway 938). The control plane VCN 1216 may include the service gateway 1236 and the NAT gateway 1238.

[0144] Data plane VCN 1218 may include data plane application layer 1246 (e.g., Figure 9 Data plane application layer 946), data plane DMZ layer 1248 (e.g., Figure 9 Data plane DMZ layer 948), and data plane data layer 1250 (e.g., Figure 9 The data plane data layer 950). The data plane DMZ layer 1248 may include one or more trusted application subnets 1260 that can be communicatively coupled to the data plane application layer 1246 (e.g., Figure 11 (one or more) trusted application subnets 1160 and (one or more) untrusted application subnets 1262 (e.g., Figure 11 The data plane VCN 1250 may include one or more untrusted application subnets 1162 and one or more LB subnets 1222 of Internet gateway 1234 contained in data plane VCN 1218. One or more trusted application subnets 1260 may communicatively couple to service gateway 1236, NAT gateway 1238 contained in data plane VCN 1218, and one or more DB subnets 1230 contained in data plane data layer 1250. One or more untrusted application subnets 1262 may communicatively couple to service gateway 1236 contained in data plane VCN 1218 and one or more DB subnets 1230 contained in data plane data layer 1250. Data plane data layer 1250 may include one or more DB subnets 1230 that may communicatively couple to service gateway 1236 contained in data plane VCN 1218.

[0145] One or more untrusted application subnets 1262 may include a primary VNIC 1264(1)-(N) communicatively coupled to tenant virtual machines (VMs) 1266(1)-(N) residing within one or more untrusted application subnets 1262. Each tenant VM 1266(1)-(N) may run code in a corresponding container 1267(1)-(N) and may be communicatively coupled to an application subnet 1226 that may be contained in a data plane application layer 1246 contained in a container egress VCN 1268. A corresponding secondary VNIC 1272(1)-(N) may facilitate communication between one or more untrusted application subnets 1262 contained in a data plane VCN 1218 and the application subnet contained in a container egress VCN 1268. The container egress VCN may include a public internet 1254 (e.g., Figure 9 The public internet (954) uses NAT gateway 1238.

[0146] Internet gateway 1234, contained in control plane VCN 1216 and data plane VCN 1218, can be communicatively coupled to metadata management service 1252 (e.g., Figure 9 A metadata management system 952 is provided, which can communicatively couple to the public internet 1254. The public internet 1254 can communicatively couple to a NAT gateway 1238 contained in a control plane VCN 1216 and a data plane VCN 1218. A service gateway 1236 contained in a control plane VCN 1216 and a data plane VCN 1218 can communicatively couple to a cloud service 1256.

[0147] In some examples, by Figure 12 The architecture pattern illustrated in block diagram 1200 can be considered as being composed of... Figure 11 The architecture illustrated in block diagram 1100 is an exception to the pattern shown, and this pattern may be what the IaaS provider's customers would expect if the IaaS provider cannot communicate directly with the customer (e.g., in a disconnected region). The corresponding containers 1267(1)-(N) contained in each customer's VM 1266(1)-(N) can be accessed by the customer in real time. Containers 1267(1)-(N) can be configured to invoke corresponding auxiliary VNICs 1272(1)-(N) contained in one or more application subnets 1226 of the data plane application layer 1246, which may be contained in the container egress VCN 1268. The auxiliary VNICs 1272(1)-(N) can transmit the calls to a NAT gateway 1238, which can transmit the calls to the public internet 1254. In this example, containers 1267(1)-(N), which can be accessed by clients in real time, can be isolated from the control plane VCN 1216 and from other entities contained in the data plane VCN 1218. Containers 1267(1)-(N) can also be isolated from resources from other clients.

[0148] In other examples, a client may use containers 1267(1)-(N) to invoke cloud service 1256. In this example, the client may run code within containers 1267(1)-(N) requesting a service from cloud service 1256. Container 1267(1)-(N) may transmit the request to a secondary VNIC 1272(1)-(N), which may transmit the request to a NAT gateway, which may transmit the request to the public internet 1254. The public internet 1254 may transmit the request via internet gateway 1234 to one or more LB subnets 1222 contained in control plane VCN 1216. In response to determining that the request is valid, one or more LB subnets may transmit the request to one or more application subnets 1226, which may transmit the request to cloud service 1256 via service gateway 1236.

[0149] It should be recognized that the IaaS architectures 900, 1000, 1100, and 1200 depicted in the figures may have other components besides those depicted. Furthermore, the embodiments shown in the figures are merely some examples of cloud infrastructure systems that can be incorporated into embodiments of this disclosure. In some other embodiments, the IaaS system may have more or fewer components than shown in the figures, may combine two or more components, or may have different configurations or component arrangements.

[0150] In some embodiments, the IaaS system described herein may include application suites, middleware, and database service offerings delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such an IaaS system is Oracle Cloud Infrastructure (OCI) provided by this assignee.

[0151] Figure 13 An example computer system 1300, in which various embodiments can be implemented, is illustrated. System 1300 can be used to implement any of the computer systems described above. As shown, computer system 1300 includes a processing unit 1304 that communicates with a plurality of peripheral subsystems via a bus subsystem 1302. These peripheral subsystems may include a processing acceleration unit 1306, an I / O subsystem 1308, a storage subsystem 1318, and a communication subsystem 1324. Storage subsystem 1318 includes a tangible computer-readable storage medium 1322 and system memory 1310.

[0152] Bus subsystem 1302 provides a mechanism for enabling various components and subsystems of computer system 1300 to communicate with each other as intended. While bus subsystem 1302 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 1302 can be any of several types of bus architectures, including memory buses or memory controllers, peripheral buses, and local buses using any of the various bus architectures. For example, such architectures may include Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses, which may be implemented as Mezzanine buses manufactured according to the IEEE P1386.1 standard.

[0153] A processing unit 1304, which may be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of the computer system 1300. One or more processors may be included in the processing unit 1304. These processors may include single-core or multi-core processors. In some embodiments, the processing unit 1304 may be implemented as one or more independent processing units 1332 and / or 1334, wherein each processing unit includes a single-core or multi-core processor. In other embodiments, the processing unit 1304 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0154] In various embodiments, processing unit 1304 can execute various programs in response to program code and can maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed can reside in processor(s) 1304 and / or storage subsystem 1318. With appropriate programming, processor(s) 1304 can provide the various functions described above. Computer system 1300 may additionally include processing acceleration unit 1306, which may include digital signal processor (DSP), dedicated processor, etc.

[0155] I / O subsystem 1308 may include user interface input devices and user interface output devices. User interface input devices may include keyboards, pointing devices such as mice or trackballs, touchpads or touchscreens integrated into a display, scroll wheels, click wheels, dials, buttons, switches, keyboards, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may include, for example, motion sensing and / or gesture recognition devices, such as the motion sensor of Microsoft Kinect®, which enables users to control and interact with input devices (such as game controllers for the Microsoft Xbox® 360) via a natural user interface using gestures and voice commands. User interface input devices may also include eye posture recognition devices, such as the Google Glass® blink detector, which detects eye activity from the user (e.g., “blinking” when taking a photo and / or making menu selections) and translates the eye posture into input in an input device (e.g., Google Glass®). Additionally, user interface input devices may include voice recognition sensing devices that enable users to interact with a voice recognition system (e.g., the Siri® navigator) via voice commands.

[0156] User interface input devices may also include, but are not limited to, 3D mice, joysticks or pointing sticks, game panels and drawing tablets, as well as audio / video devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. Additionally, user interface input devices may include, for example, medical imaging input devices such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and medical ultrasound equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, etc.

[0157] User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices, etc. Display subsystems may be cathode ray tubes (CRTs), flat panel devices such as those using liquid crystal displays (LCDs) or plasma displays, projection devices, touchscreens, etc. Generally, the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from computer system 1300 to a user or other computer. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0158] Computer system 1300 may include storage subsystem 1318 that provides a tangible, non-transitory, computer-readable storage medium for storing software and data constructs that provide the functionality of the embodiments described in this disclosure. The software may include programs, code, instructions, scripts, etc., which, when executed by one or more cores or processors of processing unit 1304, provide the aforementioned functionality. Storage subsystem 1318 may also provide a repository for storing data used according to this disclosure.

[0159] like Figure 13 As depicted in the example, storage subsystem 1318 may include various components, including system memory 1310, computer-readable storage medium 1322, and computer-readable storage medium reader 1320. System memory 1310 may store program instructions that can be loaded and executed by processing unit 1304. System memory 1310 may also store data used during instruction execution and / or data generated during program instruction execution. Various types of programs may be loaded into system memory 1310, including but not limited to client applications, web browsers, middleware applications, relational database management systems (RDBMS), virtual machines, containers, etc.

[0160] System memory 1310 may also store operating system 1316. Examples of operating system 1316 may include various versions of Microsoft Windows®, Apple Macintosh® and / or Linux operating systems, various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.) and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® OS, and Palm® OS. In some embodiments where computer system 1300 executes one or more virtual machines, the virtual machine and its guest operating system (GOS) may be loaded into system memory 1310 and executed by one or more processors or cores of processing unit 1304.

[0161] Depending on the type of computer system 1300, system memory 1310 may have different configurations. For example, system memory 1310 may be volatile memory (such as random access memory (RAM)) and / or non-volatile memory (such as read-only memory (ROM), flash memory, etc.). Different types of RAM configurations may be provided, including static random access memory (SRAM), dynamic random access memory (DRAM), etc. In some implementations, system memory 1310 may include a basic input / output system (BIOS) containing basic routines that facilitate, for example, the transfer of information between components within computer system 1300 during startup.

[0162] Computer-readable storage medium 1322 may represent a remote, local, fixed and / or removable storage device and a storage medium for temporarily and / or more permanently containing and storing computer-readable information (including instructions executable by the processing unit 1304 of the computer system 1300) for use by the computer system 1300.

[0163] Computer-readable storage medium 1322 may include any suitable medium known or used in the art, including storage media and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented in any way or technique for storing and / or transmitting information. This may include tangible computer-readable storage media such as RAM, ROM, electrically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or other tangible computer-readable media.

[0164] As an example, computer-readable storage medium 1322 may include a hard disk drive that reads or writes to a non-removable non-volatile magnetic medium, a disk drive that reads or writes to a removable non-volatile magnetic disk, and an optical disc drive that reads or writes to a removable non-volatile optical disc (such as a CD-ROM, DVD, and Blu-ray® disc or other optical media). Computer-readable storage medium 1322 may include, but is not limited to, Zip® drives, flash memory cards, Universal Serial Bus (USB) flash memory drives, Secure Digital (SD) cards, DVD discs, digital video tapes, and so on. Computer-readable storage medium 1322 may also include solid-state drives (SSDs) based on non-volatile memory (such as flash memory-based SSDs, enterprise flash drives, solid-state ROMs, etc.), volatile memory-based SSDs (such as solid-state RAM, dynamic RAM, static RAM), DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs using a combination of DRAM-based and flash memory-based SSDs. Disk drives and their associated computer-readable media can provide non-volatile storage for computer-readable instructions, data structures, program services and other data for computer system 1300.

[0165] Machine-readable instructions executable by one or more processors or cores of the processing unit 1304 may be stored on a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium may include physically tangible memory or storage devices, including volatile memory storage devices and / or non-volatile memory devices. Examples of non-transitory computer-readable storage media include magnetic storage media (e.g., disks or tapes), optical storage media (e.g., DVDs, CDs), various types of RAM, ROM, or flash memory, hard disk drives, floppy disk drives, removable memory drives (e.g., USB drives), or other types of storage devices.

[0166] The communication subsystem 1324 provides an interface to other computer systems and networks. The communication subsystem 1324 serves as an interface for receiving data from other systems and sending data from computer system 1300 to other systems. For example, the communication subsystem 1324 may enable computer system 1300 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 1324 may include radio frequency (RF) transceiver components (e.g., advanced data network technologies using cellular telephone technologies, such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), Wi-Fi (IEEE 802.11 series standards), or other mobile communication technologies, or any combination thereof), a global positioning system (GPS) receiver component, and / or other components for accessing wireless voice and / or data networks. In some embodiments, as an addition to or alternative to the wireless interface, the communication subsystem 1324 may provide wired network connectivity (e.g., Ethernet).

[0167] In some embodiments, the communication subsystem 1324 may also represent one or more users who can use the computer system 1300 to receive input communications in the form of structured and / or unstructured data feeds 1326, event streams 1328, event updates 1330, etc.

[0168] As an example, the communication subsystem 1324 may be configured to receive data feeds 1326 in real time from users of social networks and / or other communication services, such as Twitter® feeds, Facebook® updates, web feeds (such as rich site summary (RSS) feeds), and / or real-time updates from one or more third-party information sources.

[0169] Additionally, the communication subsystem 1324 can also be configured to receive data in the form of a continuous data stream, which may include event streams 1328 and / or event updates 1330 that are essentially continuous or unbounded real-time events without a clearly defined termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial quotation machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, and so on.

[0170] The communication subsystem 1324 can also be configured to output structured and / or unstructured data feeds 1326, event streams 1328, event updates 1330, etc. to one or more databases, which can communicate with one or more streaming data source computers coupled to the computer system 1300.

[0171] The computer system 1300 can be of a variety of types, including handheld portable devices (e.g., iPhone® cellular phones, iPad® computing tablets, PDAs), wearable devices (e.g., Google Glass® head-mounted displays), PCs, workstations, mainframes, information stations, server racks, or any other data processing systems.

[0172] Due to the ever-evolving nature of computers and networks, the description of the computer system 1300 depicted in the figures is merely a concrete example. Many other configurations with more or fewer components than the system depicted in the figures are possible. For example, custom hardware may be used and / or specific elements may be implemented using hardware, firmware, software (including applets), or a combination thereof. Additionally, connections to other computing devices, such as network input / output devices, may also be employed. Based on the disclosure and teachings provided herein, those skilled in the art will recognize other ways and / or methods for implementing the various embodiments.

[0173] While specific embodiments have been described, various modifications, alterations, alternative constructions, and equivalents are also covered within the scope of this disclosure. The embodiments are not limited to operation within certain specific data processing environments, but can operate freely within multiple data processing environments. Additionally, although the embodiments have been described using a specific series of transactions and steps, those skilled in the art will understand that the scope of this disclosure is not limited to the described series of transactions and steps. Various features and aspects of the above embodiments can be used individually or in combination.

[0174] Furthermore, while embodiments have been described using specific combinations of hardware and software, it should be recognized that other combinations of hardware and software are also within the scope of this disclosure. Embodiments may be implemented using only hardware, or only software, or a combination thereof. The various processes described herein can be implemented in any combination on the same processor or on different processors. Accordingly, where a component or module is described as being configured to perform certain operations, such configuration can be accomplished, for example, by designing electronic circuits to perform operations, by programming programmable electronic circuits (such as microprocessors), or any combination thereof. Processes may communicate using a variety of technologies, including but not limited to conventional technologies for inter-process communication, and different pairs of processes may use different technologies, or the same pair of processes may use different technologies at different times.

[0175] Accordingly, the specification and drawings are to be considered illustrative rather than restrictive. However, it is clear that additions, omissions, deletions, and other modifications and changes may be made thereto without departing from the broader spirit and scope set forth in the claims. Therefore, while specific disclosed embodiments have been described, they are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims.

[0176] In the context of describing the disclosed embodiments (particularly in the context of the following claims), the terms “a,” “an,” and “the,” and similar designations, are to be interpreted as covering both singular and plural, unless otherwise indicated herein or clearly contradicted by the context. Unless otherwise stated, the terms “comprising,” “having,” “including,” and “containing” are to be interpreted as open-ended terms (i.e., meaning “including but not limited to”). The term “connected” should be interpreted as partially or wholly contained, attached to, or joined together, even if something intervenes in between. Unless otherwise indicated herein, the enumeration of value ranges herein is intended only as a shorthand method for individually referencing each individual value falling within that range, and each individual value is incorporated into the specification as if it were individually enumerated herein. Unless otherwise indicated herein or clearly contradicted by the context, all methods described herein can be performed in any suitable order. The use of any and all examples or exemplary language (e.g., “such as”) provided herein is intended only to better illustrate the embodiments and does not constitute a limitation on the scope of this disclosure, unless otherwise stated. No language in the specification should be construed as indicating that any unclaimed element is essential to the practice of this disclosure.

[0177] Unless otherwise expressly stated, the extractive language, such as the phrase “at least one of X, Y, or Z”, is intended to be understood within the context generally used to represent items, terms, etc., and may be X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Therefore, such extractive language is generally not intended to, and should not, imply that some embodiments require at least one of X, at least one of Y, or at least one of Z to be present respectively.

[0178] This document describes preferred embodiments of the present disclosure, including the best modes known for carrying out the present disclosure. Variations of those preferred embodiments will become clear to those skilled in the art upon reading the foregoing description. Those skilled in the art should be able to suitably employ such variations and may practice the present disclosure in ways other than those specifically described herein. Accordingly, the present disclosure includes all modifications and equivalents to the subject matter recited in the appended claims, where permitted by applicable law. Furthermore, unless otherwise indicated herein, the present disclosure covers any combination of the foregoing elements in all its possible variations.

[0179] All references cited in this article, including publications, patent applications and patents, are incorporated into this article by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and elaborated in full in this article.

[0180] In the foregoing specification, various aspects of this disclosure have been described with reference to specific embodiments thereof; however, those skilled in the art will recognize that this disclosure is not limited thereto. The various features and aspects of the foregoing disclosure may be used individually or in combination. Furthermore, embodiments may be used in any number of settings and applications other than those described herein without departing from the broader spirit and scope of this specification. Accordingly, this specification and the accompanying drawings should be considered illustrative rather than restrictive.

Claims

1. A method comprising: Implement a certificate service data plane on computing equipment in a data center with a smaller footprint; At the Certificate Service data plane, a certificate signing request is received from an agent executing on an additional computing device communicatively connected to the computing device. The certificate signing request includes identity information corresponding to the agent's identity. Authentication information is obtained from the certificate service data plane; The certificate service data plane verifies the agent's identity by comparing the identity information with the authentication information at least once. as well as The certificate service data plane issues signed certificates to agents.

2. The method of claim 1, wherein the additional computing device includes a smart network interface card.

3. The method of claim 2, wherein the intelligent network interface card is connected to a server device in a data center with a reduced footprint.

4. The method of claim 1, wherein the additional computing device includes an integrated unattended manager.

5. The method of claim 1, wherein obtaining authentication information includes obtaining authentication information from a trusted platform module connected to the computing device.

6. The method of claim 5, wherein authentication information is supplied on the trusted platform module during the startup operation of the reduced-footprint data center.

7. The method of claim 5, wherein the authentication information includes a root certificate corresponding to a root certificate authority.

8. A computing system, comprising: One or more processors; as well as One or more memories storing computer-executable instructions that, when executed by the one or more processors, cause the computing system to at least: Implement a certificate service data plane on computing equipment in a data center with a smaller footprint; At the Certificate Service data plane, a certificate signing request is received from an agent executing on an additional computing device communicatively connected to the computing device. The certificate signing request includes identity information corresponding to the agent's identity. Authentication information is obtained from the certificate service data plane; The certificate service data plane verifies the agent's identity by comparing the identity information with the authentication information at least once. as well as The certificate service data plane issues signed certificates to agents.

9. The computing system of claim 8, wherein the additional computing device includes a smart network interface card.

10. The computing system of claim 9, wherein the intelligent network interface card is connected to a server device in a data center with a reduced footprint.

11. The computing system of claim 8, wherein the additional computing device includes an integrated unattended manager.

12. The computing system of claim 8, wherein obtaining authentication information includes obtaining authentication information from a trusted platform module connected to the computing device.

13. The computing system of claim 12, wherein authentication information is supplied on a trusted platform module during startup operations of a space-saving data center.

14. The computing system of claim 12, wherein the authentication information includes a root certificate corresponding to a root certificate authority.

15. A non-transitory computer-readable medium storing instructions, which, when executed by one or more processors of a computing system, cause the computing system to at least: Implement a certificate service data plane on computing equipment in a data center with a smaller footprint; At the Certificate Service data plane, a certificate signing request is received from an agent executing on an additional computing device communicatively connected to the computing device. The certificate signing request includes identity information corresponding to the agent's identity. Authentication information is obtained from the certificate service data plane; The certificate service data plane verifies the agent's identity by comparing the identity information with the authentication information at least once. as well as The certificate service data plane issues signed certificates to agents.

16. The non-transitory computer-readable medium of claim 16, wherein the additional computing device includes a smart network interface card.

17. The non-transitory computer-readable medium of claim 17, wherein the intelligent network interface card is connected to a server device in a data center with a reduced footprint.

18. The non-transitory computer-readable medium of claim 16, wherein the additional computing device includes an integrated unattended manager.

19. The non-transitory computer-readable medium of claim 16, wherein obtaining authentication information includes obtaining authentication information from a trusted platform module connected to the computing device.

20. The non-transitory computer-readable medium of claim 19, wherein authentication information is supplied on a trusted platform module during startup operations of a space-saving data center.

Images