Updating digital certificates associated with virtual cloud networks

Abstract

Technologies for updating a certificate bundle may include receiving, at an entity associated with a virtual cloud network, a certificate bundle including a set of updated certificate authority (CA) certificates. The techniques may include applying a verification process to an entity certificate based on the certificate bundle package, the entity certificate being issued to the entity before the entity receives the certificate bundle package. The verification process may include verifying, by the entity, a certificate chain including the entity certificate and CA certificates included in the set of updated CA certificates. The techniques may include, in response to verifying the certificate chain, installing a certificate bundle package in a storage medium associated with the entity, and authenticating, by the entity, at least one additional entity associated with the virtual cloud network using the certificate bundle package.

Description

Technical Field

[0001] This disclosure relates to digital certificates used by network entities to authenticate other network entities. Specifically, this disclosure relates to updating digital certificates and distributing updated digital certificates to network entities associated with computer networks. Background Technology

[0002] A virtual cloud network comprises network entities that communicate with each other. Communication between network entities can be performed according to security protocols, whereby network entities authenticate each other by presenting digital certificates. Digital certificates can be issued to network entities by Certificate Authorities (CAs). A digital certificate may include the CA's digital signature, which other network entities can validate using the CA's public key available to those other network entities. When a network entity presents a valid digital certificate to another network entity, that other network entity can trust that it is communicating with that network entity and not some unknown entity based on its trust relationship with the CA that issued the digital certificate.

[0003] The content of this background section should not be construed as prior art simply because it appears in this section. Attached Figure Description

[0004] The accompanying figures illustrate embodiments by way of example and not limitation. It should be noted that the terms "a" or "an" embodiment used in this disclosure do not necessarily refer to the same embodiment, and mean at least one. In the figures:

[0005] Figures 1 to 4 This is a block diagram illustrating a pattern for implementing a cloud infrastructure as a service system according to one or more embodiments;

[0006] Figure 5 It is a hardware system according to one or more embodiments;

[0007] Figure 6A and 6B Example virtual cloud networks according to one or more embodiments are illustrated respectively;

[0008] Figure 7 This is a flowchart illustrating example operations involving updating a CA certificate according to one or more embodiments;

[0009] Figures 8A to 8D This further illustrates a timeline of example operations involving the distribution of certificate bundles to network entities associated with a virtual cloud network, according to one or more embodiments; and

[0010] Figures 9A to 9CThis is a flowchart further illustrating example operations involving the distribution of certificate bundles to network entities associated with a virtual cloud network, according to one or more embodiments. Detailed Implementation

[0011] In the following description, numerous specific details are set forth for ease of explanation in order to provide a thorough understanding. One or more embodiments may be practiced without these specific details. Features described in one embodiment may be combined with features described in different embodiments. In some examples, well-known structures and devices are described in the form of block diagrams to avoid unnecessarily obscuring the invention.

[0012] 1. Introduction

[0013] 2. General Overview

[0014] 3. Cloud computing technology

[0015] 4. Computer System

[0016] 5. System Architecture Overview

[0017] 6. Authenticating network entities

[0018] 7. Renew Certificate Authority Certificates

[0019] 8. Distribute the certificate bundle

[0020] A. Distribution Operation Timeline

[0021] B. Distribution Operation Processing

[0022] 9. Other matters; extension

[0023] 1. Introduction

[0024] A system generates certificate bundles and distributes them to network entities associated with a virtual cloud network. The certificate bundles may include a set of CA certificates that can be used by the network entities to authenticate other network entities associated with the virtual cloud network. The CA certificates in the certificate bundles include various CAs that have issued entity certificates to various network entities associated with the virtual cloud network. When a network entity presents an entity certificate to another entity, that other entity can authenticate the entity certificate by obtaining the public key corresponding to the CA that issued the entity certificate from the certificate bundle and using the public key corresponding to that CA to decrypt the digital signature on the entity certificate.

[0025] 2. General Overview

[0026] One or more embodiments apply verification processing to verify a set of updated CA certificates before relying on that set to authenticate other network entities. This verification processing may be referred to herein as self-verification processing because it involves entities verifying their own entity certificates.

[0027] The system distributes an updated certificate bundle, including a set of updated CA certificates, to a group of network entities. When a specific network entity receives the updated certificate bundle, it can verify the updated certificate bundle by applying a self-verification process to its own current entity certificate based on the updated certificate bundle. The entity certificate used in the verification process performed by the specific network entity is the entity certificate issued to that specific network entity before it received the updated certificate bundle. If the self-verification process is successful, the specific network entity accepts the updated certificate bundle and begins using the updated CA certificates in the updated certificate bundle to authenticate other network entities. If the self-verification process is unsuccessful, the specific network entity rejects the updated certificate bundle and avoids using the updated CA certificates in the updated certificate bundle to authenticate other network entities. Instead, after rejecting the updated certificate bundle, a particular network entity may use the CA certificate in the current certificate bundle to authenticate other network entities, which (a) is currently being used to authenticate other network entities and (b) was distributed to that particular network entity before the updated certificate bundle.

[0028] A network entity can determine that the self-verification process was successful when (a) the updated certificate bundle includes a CA certificate used to sign its own entity certificate, or (b) a certificate chain that begins with the network entity's own certificate and ends with a CA certificate included in the updated certificate bundle is verified by the network entity. As used herein, the term "certificate chain" includes a sequence of digital certificates having a hierarchical structure starting with an entity certificate and ending with a top-level, root CA certificate, wherein each digital certificate in the sequence is issued by a CA identified in the next digital certificate in the hierarchy, except for the root CA certificate, which is self-issued by a CA identified in the root CA certificate.

[0029] In one example, a certificate chain can be verified by: (a) matching the public key of the signing CA certificate used by the CA to sign the current certificate with the digital signature of the CA on the current certificate; and (b) repeating the matching operation using the signing CA certificate as the current certificate until the digital signature of each current certificate has matched the public key of the signing CA certificate. Multiple intermediate CA certificates can be included in the certificate chain, between the entity certificate of a particular network entity and the root CA certificate at the end of the chain included in the updated set of CA certificates.

[0030] In one example, a particular network entity may determine that the self-verification process failed when (a) its own entity certificate has not been signed using any of the CA certificates included in the updated set of CA certificates, and (b) the certificate chain beginning with the particular network entity's own certificate does not include any of the CA certificates included in the updated set of CA certificates. Additionally, or alternatively, a particular network entity may determine that the self-verification process failed when any certificate in the certificate chain beginning with the particular network entity's own entity certificate is not verified. For example, if the digital signature of the CA on the certificate does not match the CA's public key, then the certificate may be invalidated.

[0031] To transition a group of network entities from a current first set of CA certificates to an updated second set of CA certificates, one or more embodiments first distribute an intermediate certificate bundle comprising a set of updated CA certificates, followed by a second certificate bundle. The set of updated CA certificates in the intermediate certificate bundle includes a first subset of CA certificates still in use by the network entities, and a second subset of new CA certificates intended for future use by the network entities. The system distributes the intermediate certificate bundle to the network entities while they are using the current first certificate bundle to authenticate other network entities. A specific network entity accepts the intermediate certificate bundle after verifying its current entity certificate at least partially based on the CA certificates in the intermediate certificate bundle. Subsequently, the system provides the specific network entity with a new entity certificate replacing its current entity certificate. The new entity certificate is signed (directly or indirectly via a certificate chain) by a CA corresponding to the CA certificates included in the set of updated CA certificates in the intermediate certificate bundle. The system then distributes a second certificate bundle, which (a) includes a subset of new CA certificates from the intermediate certificate bundle, and (b) does not include at least one CA certificate from the first subset of CA certificates previously included in the intermediate certificate bundle. A specific network entity accepts the second certificate bundle after verifying its new entity certificate at least in part based on the CA certificates included in the second certificate bundle using the verification process described herein (verification based on the CA certificates in the certificate bundle, or verification by confirming the validity of the certificate chain).

[0032] One or more embodiments provide improvements to the technology associated with the distribution of new CA certificates used in a virtual cloud network. In one improvement to this technology, the currently disclosed technology eliminates the need for downtime when updating CA certificates. Downtime is avoided through a certificate bundle distribution process that, among other things, ensures by design that the root certificate in each certificate bundle distributed to a group of network entities corresponds to each entity certificate of each network entity, and further avoids downtime by practically ensuring compliance with the verification processing of the certificate bundle distribution process described above. If at any time a network entity's entity certificate is issued by a CA certificate not in a certificate bundle that is installed on a group of network entities, then such a network entity will be unable to perform the necessary authentication process to establish communication with other network entities, potentially leading to widespread downtime across the virtual cloud network. By performing verification processing on the updated certificate bundle as described herein, the network entity confirms that other entities will be able to authenticate the network entity using the updated certificate bundle when the network entity presents its entity certificate for authentication. Similarly, other network entities verifying the updated certificate bundle confirm that the network entity will be able to authenticate them using the updated certificate bundle when they present their entity certificates for authentication. Therefore, potential problems specific to a particular network entity can be identified in the verification processor before earlier CA certificates are removed.

[0033] The distribution of intermediate certificate bundles (which include both the new and older CA certificates) avoids downtime because network entities perform a first self-verification process before installing the intermediate certificate bundle. This first self-verification process guarantees that, when the intermediate certificate bundle is installed, the older CA certificate of the CA that issued the earlier entity certificate for that network entity can still be successfully used by other entities to authenticate the entity certificate of that network entity. Similarly, the distribution of second certificate bundles (which include the new CA certificate, and the older CA certificate has been removed) avoids downtime because network entities perform a second self-verification process before installing the second certificate bundle. This second self-verification process guarantees that, when the second certificate bundle is installed, the new CA certificate of the CA that issued the new entity certificate for that network entity can be successfully used by other entities to authenticate the new entity certificate of that network entity. Furthermore, contrary to currently disclosed techniques, distributing certificate bundles that only include the new CA certificate causes downtime until all network entities have eventually updated. Additionally, incorporating CA certificates into operating system (OS) images generally requires downtime when transitioning from an older OS image to a new OS image that includes the new CA certificate. By utilizing an updated certificate bundle to distribute new CA certificates, downtime associated with the transition between OS images is avoided.

[0034] Furthermore, one or more embodiments improve virtual cloud network provisioning technology. For example, the distribution of updated certificate bundles is faster than generating updated OS images, utilizes fewer computing resources, and avoids downtime associated with transitioning to updated OS images. When provisioning a region of a virtual cloud network, for example, CA certificates can be updated multiple times during the provisioning process according to security protocols that require CA certificate updates. Therefore, using these security protocols, combining CA certificates into an OS image and generating updated OS images sequentially with each CA certificate update can be impractical. For example, the time consumed in generating updated OS images for each updated set of CA certificates would exceed the time constraints of provisioning a virtual cloud network. Accordingly, by distributing new CA certificates in updated certificate bundles as described herein, new CA certificates are distributed more rapidly while also ensuring zero downtime.

[0035] One or more embodiments described in this specification and / or set forth in the claims may not be included in this general overview section.

[0036] 3. Cloud computing technology

[0037] Infrastructure as a Service (IaaS) is an application of cloud computing technology. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, cloud providers can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, IaaS providers can also offer various services accompanying these infrastructure components (example services include billing software, monitoring software, logging software, load balancing software, clustering software, etc.). Therefore, because these services may be policy-driven, IaaS users can implement policies to drive load balancing to maintain application availability and performance.

[0038] In some cases, IaaS customers can access resources and services over a wide area network (WAN) such as the Internet and can use the cloud provider's services to install the remaining elements of the application stack. For example, a user can log in to the IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create buckets for workloads and backups, and even install enterprise software into that VM. The customer can then use the provider's services to perform various functions, including balancing network traffic, troubleshooting application issues, monitoring performance, and managing disaster recovery.

[0039] In some cases, cloud computing models will involve the involvement of cloud providers. Cloud providers can, but are not necessarily, third-party providers specializing in (e.g., provisioning, renting, selling) IaaS services. Entities may also choose to deploy private clouds, thus becoming their own infrastructure service providers.

[0040] In some examples, IaaS deployment involves the process of implementing a new application or a new version of an application on a prepared application server or other similar device. IaaS deployment may also include the processing of server preparation (e.g., installation libraries, daemons, etc.). Deployment processing is typically managed by the cloud provider and resides below the hypervisor layer (e.g., servers, storage devices, network hardware, and virtualization). Therefore, the customer can be responsible for processing (OS), middleware, and / or application deployment (e.g., on self-service virtual machines, etc., which can be started on demand).

[0041] In some examples, IaaS provisioning can refer to acquiring computers or virtual hosts for use, or even installing necessary libraries or services on them. In most cases, deployment does not include provisioning, and provisioning may need to be performed first.

[0042] In some cases, IaaS provisioning presents challenges. The initial challenge lies in provisioning an initial set of infrastructure. After the initial provisioning is complete, the evolution of the existing infrastructure (e.g., adding new services, changing services, removing services, etc.) also presents additional challenges. In some cases, both challenges can be addressed by enabling the declarative definition of infrastructure configuration. In other words, the infrastructure (e.g., which components are needed and how they interact) can be defined by one or more configuration files. Therefore, the overall topology of the infrastructure (e.g., which resources depend on which resources and how they work together) can be described declaratively. In some cases, once the topology is defined, workflows for creating and / or managing the different components described in the configuration files can be generated.

[0043] In some examples, the infrastructure can have many interconnected elements. For example, there may be one or more Virtual Private Clouds (VPCs) (e.g., potential on-demand pools of configurable and / or shared computing resources), also known as the core network. In some examples, there may also be one or more inbound / outbound traffic group rules provisioned to define how inbound and / or outbound traffic will be configured for the network, as well as one or more Virtual Machines (VMs). Other infrastructure elements, such as load balancers, databases, etc., may also be provisioned. The infrastructure can evolve incrementally as more and / or additional infrastructure elements are desired.

[0044] In some cases, continuous deployment techniques can be employed to enable the deployment of infrastructure code across various virtual computing environments. Furthermore, the described techniques enable infrastructure management within these environments. In some examples, service teams may write code that they wish to deploy to one or more, but often many, different production environments (e.g., across various geographical locations, sometimes spanning the entire world). In some embodiments, infrastructure and resources can be provisioned (manually and / or using provisioning tools) before deploying the code to be executed on the infrastructure. However, in some examples, the infrastructure on which the code will be deployed must first be established. In some cases, provisioning can be done manually, resources can be provisioned using provisioning tools, and / or once the infrastructure is provisioned, code can be deployed using deployment tools.

[0045] Figure 1 This is a block diagram illustrating an example pattern 100 of an IaaS architecture according to at least one embodiment. Service provider 102 may be communicatively coupled to secure hosting lease 104, which may include a virtual cloud network (VCN) 106 and a secure hosting subnet 108. In some examples, service provider 102 may use one or more client computing devices, which may be portable handheld devices (e.g., iPhone®, cellular phone, iPad®, computing tablet, personal digital assistant (PDA)) or wearable devices (e.g., Google Glass® head-mounted display), running software such as Microsoft Windows Mobile® and / or various mobile operating systems such as iOS, Windows Phone, Android, BlackBerry 8, PalmOS, and supporting the Internet, email, short message service (SMS), Blackberry®, or other communication protocols. Alternatively, client computing devices may be general-purpose personal computers, including, for example, personal computers and / or laptops running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems. The client computing device can be a workstation computer running any of a variety of commercial UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, such as, for example, Google Chrome OS). Alternatively, or additionally, the client computing device can be any other electronic device, such as a thin client computer, an internet-enabled gaming system (e.g., a Microsoft Xbox game console with or without Kinect® gesture input), and / or a personal messaging device capable of communicating over a network that can access VCN 106 and / or the internet.

[0046] VCN 106 may include a local peering gateway (LPG) 110, which may be communicatively coupled to a secure shell (SSH) VCN 112 via an LPG 110 contained in an SSH VCN 112. SSH VCN 112 may include an SSH subnet 114, and SSH VCN 112 may be communicatively coupled to a control plane VCN 116 via an LPG 110 contained in a control plane VCN 116. Furthermore, SSH VCN 112 may be communicatively coupled to a data plane VCN 118 via an LPG 110. Control plane VCN 116 and data plane VCN 118 may be contained within a service lease 119 that may be owned and / or operated by an IaaS provider.

[0047] The control plane VCN 116 may include a control plane demilitarized zone (DMZ) layer 120 that acts as a peripheral network (e.g., a portion of a corporate network between an intranet and an external network). DMZ-based servers can assume limited liability and help control vulnerabilities. Furthermore, the DMZ layer 120 may include one or more load balancer (LB) subnets 122, a control plane application layer 124 that may include one or more application subnets 126, and a control plane data layer 128 that may include one or more database (DB) subnets 130 (e.g., one or more front-end DB subnets and / or one or more back-end DB subnets). One or more LB subnets 122 contained in the control plane DMZ layer 120 may be communicatively coupled to one or more application subnets 126 contained in the control plane application layer 124 and an Internet gateway 134 that may be contained in the control plane VCN 116, and one or more application subnets 126 may be communicatively coupled to one or more DB subnets 130 contained in the control plane data layer 128, as well as a service gateway 136 and a network address translation (NAT) gateway 138. The control plane VCN 116 may include the service gateway 136 and the NAT gateway 138.

[0048] The control plane VCN 116 may include a data plane mirror application layer 140, which may include one or more application subnets 126. The one or more application subnets 126 contained in the data plane mirror application layer 140 may include a virtual network interface controller (VNIC) 142 that can execute a compute instance 144. The compute instance 144 may communicatively couple the one or more application subnets 126 of the data plane mirror application layer 140 to the one or more application subnets 126 that may be contained in the data plane application layer 146.

[0049] Data plane VCN 118 may include data plane application layer 146, data plane DMZ layer 148, and data plane data layer 150. Data plane DMZ layer 148 may include one or more application subnets 126 communicatively coupled to data plane application layer 146 and one or more LB subnets 122 communicatively coupled to Internet gateway 134 of data plane VCN 118. One or more application subnets 126 may be communicatively coupled to service gateway 136 and NAT gateway 138 of data plane VCN 118. Data plane data layer 150 may also include one or more DB subnets 130 communicatively coupled to one or more application subnets 126 of data plane application layer 146.

[0050] The Internet gateway 134 of the control plane VCN 116 and data plane VCN 118 can be communicatively coupled to the metadata management service 152, which in turn can be communicatively coupled to the public Internet 154. The public Internet 154 can be communicatively coupled to the NAT gateway 138 of the control plane VCN 116 and data plane VCN 118. The service gateway 136 of the control plane VCN 116 and data plane VCN 118 can be communicatively coupled to the cloud service 156.

[0051] In some examples, the service gateway 136 of the control plane VCN 116 or data plane VCN 118 can make application programming interface (API) calls to the cloud service 156 without traversing the public internet 154. API calls from the service gateway 136 to the cloud service 156 can be unidirectional: the service gateway 136 can make API calls to the cloud service 156, and the cloud service 156 can send requested data to the service gateway 136. However, the cloud service 156 may not initiate API calls to the service gateway 136.

[0052] In some examples, secure host lease 104 can be directly connected to service lease 119, which would otherwise be isolated. Secure host subnet 108 can communicate with SSH subnet 114 via LPG 110, which enables bidirectional communication between otherwise isolated systems. Connecting secure host subnet 108 to SSH subnet 114 allows secure host subnet 108 to access other entities within service lease 119.

[0053] Control plane VCN 116 allows users of service lease 119 to establish or otherwise provision desired resources. Desired resources provisioned in control plane VCN 116 can be deployed or otherwise used in data plane VCN 118. In some examples, control plane VCN 116 can be isolated from data plane VCN 118, and the data plane mirror application layer 140 of control plane VCN 116 can communicate with the data plane application layer 146 of data plane VCN 118 via VNIC 142, which can be included in both the data plane mirror application layer 140 and the data plane application layer 146.

[0054] In some examples, users or clients of the system can make requests, such as create, read, update, or delete (CRUD) operations, via the public internet 154, which can transmit requests to the metadata management service 152. The metadata management service 152 can transmit the request to the control plane VCN 116 via internet gateway 134. The request can be received by one or more LB subnets 122 contained in the control plane DMZ layer 120. The LB subnets 122 can determine that the request is valid, and in response to this determination, they can transmit the request to one or more application subnets 126 contained in the control plane application layer 124. If the request is validated and requires a call to the public internet 154, the call to the public internet 154 can be transmitted to a NAT gateway 138 that can make calls to the public internet 154. The request may expect the stored metadata to be stored in one or more DB subnets 130.

[0055] In some examples, the data plane mirroring application layer 140 can facilitate direct communication between the control plane VCN 116 and the data plane VCN 118. For example, it may be desirable to apply configuration changes, updates, or other appropriate modifications to resources contained in the data plane VCN 118. Through VNIC 142, the control plane VCN 116 can communicate directly with the resources contained in the data plane VCN 118, and thus can perform configuration changes, updates, or other appropriate modifications.

[0056] In some embodiments, the control plane VCN 116 and data plane VCN 118 may be contained within a service lease 119. In this case, the system's users or customers may not own or operate the control plane VCN 116 or data plane VCN 118. Alternatively, the IaaS provider may own or operate both the control plane VCN 116 and data plane VCN 118, both of which may be contained within the service lease 119. This embodiment can enable network isolation, which may prevent users or customers from interacting with the resources of other users or customers. Moreover, this embodiment can allow the system's users or customers to privately store databases without relying on the public internet 154, which may not have the desired level of threat prevention for storage.

[0057] In other embodiments, one or more LB subnets 122 included in the control plane VCN 116 may be configured to receive signals from the service gateway 136. In this embodiment, the control plane VCN 116 and the data plane VCN 118 may be configured to be invoked by the IaaS provider's customers without invoking the public internet 154. The IaaS provider's customers may expect this embodiment because the database(s) used by the customer can be controlled by the IaaS provider and can be stored on a service lease 119, which can be isolated from the public internet 154.

[0058] Figure 2 This is a block diagram illustrating another example pattern 200 of an IaaS architecture according to at least one embodiment. Service operator 202 (e.g., Figure 1 Service provider 102) can communicatively couple to secure host lease 204 (e.g., Figure 1 Secure hosting lease 104), secure hosting lease 204 may include Virtual Cloud Network (VCN) 206 (e.g., Figure 1 VCN106) and Secure Host Subnet 208 (e.g., Figure 1 The secure host subnet 108). VCN 206 may include a local peering gateway (LPG) 210 (e.g., Figure 1 LPG 110), LPG 210 can be communicatively coupled to Secure Shell (SSH) VCN 212 via LPG 110 contained in SSH VCN 212 (e.g., Figure 1 SSH VCN 212 can include SSH subnet 214 (e.g., SSH VCN 212). Figure 1 SSH subnet 114), and SSH VCN 212 can be communicatively coupled to control plane VCN 216 via LPG 210 contained in control plane VCN 216 (e.g., Figure 1Control plane VCN 216). Control plane VCN 216 may be included in service lease 219 (e.g., Figure 1 In the service lease 119), and the data plane VCN 218 (e.g., Figure 1 The data plane VCN 118 may be included in a customer lease 221 that may be owned or operated by a user or customer of the system.

[0059] Control plane VCN 216 may include control plane DMZ layer 220 (e.g., Figure 1 The control plane DMZ layer 220 may include one or more LB subnets 222 (e.g., Figure 1 (one or more) LB subnets 122), may include (one or more) application subnets 226 (e.g., Figure 1 The control plane application layer 224 of (one or more) application subnets 126 (e.g., Figure 1 The control plane application layer 124) may include one or more database (DB) subnets 230 (e.g., similar to...). Figure 1 The control plane data layer 228 of (one or more) DB subnets 130) (e.g., Figure 1 The control plane data layer 128). One or more LB subnets 222 contained in the control plane DMZ layer 220 can be communicatively coupled to one or more application subnets 226 contained in the control plane application layer 224 and an Internet gateway 234 that can be contained in the control plane VCN 216 (e.g., Figure 1 Internet gateway 134), and application subnet(s) 226 can communicatively couple to DB subnet(s) 230 contained in control plane data layer 228 and service gateway 236 (e.g., Figure 1 Service gateway 136) and Network Address Translation (NAT) gateway 238 (e.g., Figure 1 (NAT gateway 138). The control plane VCN 216 may include service gateway 236 and NAT gateway 238.

[0060] The control plane VCN 216 may include a data plane mirror of the application layer 240, which may include one or more application subnets 226 (e.g., Figure 1 The data plane mirror application layer 240). One or more application subnets 226 contained in the data plane mirror application layer 240 may include executable compute instances 244 (e.g., similar to...). Figure 1The virtual network interface controller (VNIC) 242 (e.g., the VNIC of 142) of the computing instance 144. The computing instance 244 may facilitate the mirroring of the application subnet(s) 226 of the application layer 240 in the data plane and may be included in the application layer 246 in the data plane (e.g., Figure 1 Communication between one or more application subnets 226 in the data plane application layer 146 via VNIC 242 contained in the data plane mirror application layer 240 and VNIC 242 contained in the data plane application layer 246.

[0061] The Internet gateway 234 included in the control plane VCN 216 can be communicatively coupled to the metadata management service 252 (e.g., Figure 1 Metadata management service 252 can communicatively couple to the public Internet 254 (e.g., Figure 1 The public internet 254 can communicatively couple to a NAT gateway 238 included in the control plane VCN 216. The service gateway 236 included in the control plane VCN 216 can communicatively couple to a cloud service 256 (e.g., ...). Figure 1 Cloud services (156).

[0062] In some examples, data plane VCN 218 may be included in customer lease 221. In this case, the IaaS provider may provide control plane VCN 216 for each customer, and the IaaS provider may establish a unique compute instance 244 for each customer, included in service lease 219. Each compute instance 244 may allow communication between control plane VCN 216 included in service lease 219 and data plane VCN 218 included in customer lease 221. Compute instance 244 may allow resources provisioned in control plane VCN 216 included in service lease 219 to be deployed or otherwise used in data plane VCN 218 included in customer lease 221.

[0063] In other examples, an IaaS provider's customer may have a database residing in customer lease 221. In this example, control plane VCN 216 may include data plane mirror application layer 240, which may include one or more application subnets 226. Data plane mirror application layer 240 may reside in data plane VCN 218, but may not reside in data plane VCN 218. That is, data plane mirror application layer 240 may have access to customer lease 221, but may not reside in data plane VCN 218 or be owned or operated by an IaaS provider's customer. Data plane mirror application layer 240 may be configured to invoke data plane VCN 218, but may not be configured to invoke any entity contained in control plane VCN 216. Customers may expect to deploy or otherwise use resources provided in the control plane VCN 216 in the data plane VCN 218, and the data plane mirroring application layer 240 can facilitate the customer's expected deployment or other use of resources.

[0064] In some embodiments, an IaaS provider's customer can apply filters to data plane VCN 218. In this embodiment, the customer can determine what data plane VCN 218 can access, and the customer can restrict access from data plane VCN 218 to the public internet 254. The IaaS provider may not be able to apply filters or otherwise control data plane VCN 218's access to any external networks or databases. Applying filters and controls to data plane VCN 218 contained in customer lease 221 helps isolate data plane VCN 218 from other customers and the public internet 254.

[0065] In some embodiments, cloud service 256 may be invoked by service gateway 236 to access services that may not exist on public internet 254, control plane VCN 216, or data plane VCN 218. The connection between cloud service 256 and control plane VCN 216 or data plane VCN 218 may not be real-time or continuous. Cloud service 256 may reside on different networks owned or operated by an IaaS provider. Cloud service 256 may be configured to receive calls from service gateway 236 and may be configured not to receive calls from public internet 254. Some cloud services 256 may be isolated from other cloud services 256, and control plane VCN 216 may be isolated from cloud services 256 that may not be in the same region as control plane VCN 216. For example, control plane VCN 216 may be located in "Region 1," and cloud service "Deployment 1" may be located in both Region 1 and "Region 2." If the service gateway 236, contained in the control plane VCN 216 located in region 1, makes a call to deployment 1, then that call can be transmitted to deployment 1 in region 1. In this example, the control plane VCN 216 or deployment 1 in region 1 may not be communicatively coupled to deployment 1 in region 2 or may otherwise communicate.

[0066] Figure 3 This is a block diagram illustrating another example pattern 300 of an IaaS architecture according to at least one embodiment. Service operator 302 (e.g., Figure 1 Service provider 102) can communicatively couple to secure host lease 304 (e.g., Figure 1 Secure hosting rental 104), Secure hosting rental 304 may include Virtual Cloud Network (VCN) 306 (e.g., Figure 1 VCN106) and Secure Host Subnet 308 (e.g., Figure 1 The secure host subnet 108). VCN 306 may include LPG 310 (e.g., Figure 1 LPG 110), LPG 310 can be communicatively coupled to SSH VCN 312 via LPG 310 contained in SSH VCN 312 (e.g., Figure 1 SSH VCN 112). SSH VCN 312 can include SSH subnet 314 (e.g., Figure 1 SSH subnet 114), and SSH VCN 312 can be communicatively coupled to control plane VCN 316 via LPG 310 included in control plane VCN 316 (e.g., Figure 1 The control plane VCN 116) and coupled to the data plane VCN 318 via the LPG 310 contained in the data plane VCN 318 (e.g., Figure 1Data plane VCN 118). Control plane VCN 316 and data plane VCN 318 may be included in service lease 319 (e.g., Figure 1 In the service rental (119).

[0067] The control plane VCN 316 may include one or more load balancer (LB) subnets 322 (e.g., Figure 1 The control plane DMZ layer 320 of (one or more) LB subnets 122) (e.g., Figure 1 The control plane DMZ layer 120 may include one or more application subnets 326 (e.g., similar to...). Figure 1 The control plane application layer 324 of (one or more) application subnets 126 (e.g., Figure 1 The control plane application layer 124), and the control plane data layer 328, which may include (one or more) DB subnets 330, for example, Figure 1 The control plane data layer 128). One or more LB subnets 322 contained in the control plane DMZ layer 320 can be communicatively coupled to one or more application subnets 326 contained in the control plane application layer 324 and an Internet gateway 334 that may be contained in the control plane VCN 316 (e.g., Figure 1 Internet gateway 134), and application subnet 326 (one or more) can communicatively couple to DB subnet 330 (one or more) contained in control plane data layer 328 and service gateway 336 (e.g., Figure 1 The service gateway) and Network Address Translation (NAT) gateway 338 (e.g., Figure 1 (NAT gateway 138). The control plane VCN 316 may include the service gateway 336 and the NAT gateway 338.

[0068] Data plane VCN 318 may include data plane application layer 346 (e.g., Figure 1 Data plane application layer 146), data plane DMZ layer 348 (e.g., Figure 1 Data plane DMZ layer 148), and data plane data layer 350 (e.g., Figure 1The data plane data layer 150. The data plane DMZ layer 348 may include one or more trusted application subnets 360 and one or more untrusted application subnets 362 communicatively coupled to the data plane application layer 346, and one or more LB subnets 322 of the Internet gateway 334 contained in the data plane VCN 318. The one or more trusted application subnets 360 may be communicatively coupled to the service gateway 336 contained in the data plane VCN 318, the NAT gateway 338 contained in the data plane VCN 318, and one or more DB subnets 330 contained in the data plane data layer 350. The one or more untrusted application subnets 362 may be communicatively coupled to the service gateway 336 contained in the data plane VCN 318 and the one or more DB subnets 330 contained in the data plane data layer 350. The data plane data layer 350 may include one or more DB subnets 330 communicatively coupled to the service gateway 336 contained in the data plane VCN 318.

[0069] One or more untrusted application subnets 362 may include one or more primary VNICs 364(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 366(1)-(N). Each tenant VM 366(1)-(N) may be communicatively coupled to a corresponding application subnet 367(1)-(N) that may be contained in a corresponding container egress VCN 368(1)-(N), which may be contained in a corresponding customer lease 370(1)-(N). A corresponding secondary VNIC 372(1)-(N) may facilitate communication between one or more untrusted application subnets 362 contained in a data plane VCN 318 and the application subnets contained in a container egress VCN 368(1)-(N). Each container egress VCN 368(1)-(N) may include a NAT gateway 338 that can be communicatively coupled to the public Internet 354 (e.g., Figure 1 The public internet (154).

[0070] An Internet gateway 334, contained in the control plane VCN 316 and the data plane VCN 318, can be communicatively coupled to a metadata management service 352 (e.g., Figure 1 Metadata management service 152 can be communicatively coupled to the public internet 354. The public internet 354 can be communicatively coupled to a NAT gateway 338 contained in a control plane VCN 316 and a data plane VCN 318. Service gateway 336 contained in a control plane VCN 316 and a data plane VCN 318 can be communicatively coupled to a cloud service 356.

[0071] In some embodiments, the data plane VCN 318 may be integrated with the customer lease 370. Such integration may be useful or desired by the IaaS provider's customers in certain situations, such as when support may be expected during code execution. Customers may provide code that could be destructive, might communicate with other customer resources, or might otherwise cause undesirable effects. In response, the IaaS provider may determine whether to run the code provided by the customer to the IaaS provider.

[0072] In some examples, an IaaS provider's customer may grant the IaaS provider temporary network access and request functionality attached to the data plane application layer 346. The code running this functionality may execute in VMs 366(1)-(N) and may not be configured to run anywhere else on the data plane VCN 318. Each VM 366(1)-(N) may be connected to a customer lease 370. The corresponding container 371(1)-(N) contained in VMs 366(1)-(N) may be configured to run the code. In this case, dual isolation may exist (e.g., container 371(1)-(N) runs the code, where container 371(1)-(N) may be contained in at least one or more untrusted application subnets 362 containing VMs 366(1)-(N)), which can help prevent incorrect or otherwise unintended code from corrupting the IaaS provider's network or the networks of different customers. Containers 371(1)-(N) may be communicatively coupled to customer lease 370 and may be configured to transmit or receive data from customer lease 370. Containers 371(1)-(N) may not be configured to transmit or receive data from any other entity in data plane VCN 318. After the code execution is complete, the IaaS provider may terminate or otherwise dispose of containers 371(1)-(N).

[0073] In some embodiments, one or more trusted application subnets 360 may run code that may be owned or operated by an IaaS provider. In this embodiment, one or more trusted application subnets 360 may be communicatively coupled to one or more database subnets 330 and configured to perform CRUD operations in one or more database subnets 330. One or more untrusted application subnets 362 may be communicatively coupled to one or more database subnets 330, but in this embodiment, one or more untrusted application subnets may be configured to perform read operations in one or more database subnets 330. Containers 371(1)-(N) that may be contained in each customer's VM 366(1)-(N) and may run code from the customer may not be communicatively coupled to one or more database subnets 330.

[0074] In other embodiments, the control plane VCN 316 and the data plane VCN 318 may be coupled without direct communication. In this embodiment, there may be no direct communication between the control plane VCN 316 and the data plane VCN 318. However, communication can occur indirectly through at least one method. The LPG 310 may be established by an IaaS provider, which can facilitate communication between the control plane VCN 316 and the data plane VCN 318. In another example, either the control plane VCN 316 or the data plane VCN 318 may invoke the cloud service 356 via the service gateway 336. For example, an invocation of the cloud service 356 from the control plane VCN 316 may include a request for a service that can communicate with the data plane VCN 318.

[0075] Figure 4 This is a block diagram illustrating another example pattern 400 of an IaaS architecture according to at least one embodiment. Service operator 402 (e.g., Figure 1 Service provider 102) can communicatively couple to secure host lease 404 (e.g., Figure 1 Secure hosting rental 104), Secure hosting rental 404 may include Virtual Cloud Network (VCN) 406 (e.g., Figure 1 VCN106) and Secure Host Subnet 408 (e.g., Figure 1 The secure host subnet 108). VCN 406 may include LPG 410 (e.g., Figure 1 LPG 110), LPG 410 can be accessed via SSH VCN 412 (e.g., Figure 1 The LPG410 in SSH VCN 412 is communicatively coupled to SSH VCN 412. SSH VCN 412 may include SSH subnet 414 (e.g., Figure 1SSH subnet 114), and SSH VCN 412 can be communicatively coupled to control plane VCN 416 via LPG 410 contained in control plane VCN 416 (e.g., Figure 1 The control plane VCN 116) and coupled to the data plane VCN 418 via LPG410 contained in the data plane VCN 418 (e.g., Figure 1 Data plane VCN 418). Control plane VCN 416 and data plane VCN 418 may be included in service lease 419 (e.g., Figure 1 In the service rental (119).

[0076] The control plane VCN 416 may include one or more LB subnets 422 (e.g., Figure 1 The control plane DMZ layer 420 of (one or more) LB subnets 122) (e.g., Figure 1 The control plane DMZ layer 120), may include (one or more) application subnets 426 (e.g., Figure 1 The control plane application layer 424 of (one or more) application subnets 126 (e.g., Figure 1 The control plane application layer 124) may include one or more DB subnets 430 (e.g., Figure 3 The control plane data layer 428 of (one or more) DB subnets 330 (e.g., Figure 1 The control plane data layer 128). One or more LB subnets 422 contained in the control plane DMZ layer 420 can be communicatively coupled to one or more application subnets 426 contained in the control plane application layer 424 and an Internet gateway 434 that may be contained in the control plane VCN 416 (e.g., Figure 1 Internet gateway 134), and application subnet 426 (one or more) can communicatively couple to DB subnet 430 (one or more) contained in control plane data layer 428 and service gateway 436 (e.g., Figure 1 The service gateway) and Network Address Translation (NAT) gateway 438 (e.g., Figure 1 (NAT gateway 138). The control plane VCN 416 may include the service gateway 436 and the NAT gateway 438.

[0077] Data plane VCN 418 may include data plane application layer 446 (e.g., Figure 1 Data plane application layer 146), data plane DMZ layer 448 (e.g., Figure 1 Data plane DMZ layer 148), and data plane data layer 450 (e.g., Figure 1The data plane data layer 150). The data plane DMZ layer 448 may include one or more trusted application subnets 460 that can be communicatively coupled to the data plane application layer 446 (e.g., Figure 3 (one or more) trusted application subnets 360 and (one or more) untrusted application subnets 462 (e.g., Figure 3 The data plane includes one or more untrusted application subnets 362 and one or more LB subnets 422 of an Internet gateway 434 contained in data plane VCN 418. One or more trusted application subnets 460 may communicatively couple to a service gateway 436 contained in data plane VCN 418, a NAT gateway 438 contained in data plane VCN 418, and one or more DB subnets 430 contained in data plane data layer 450. One or more untrusted application subnets 462 may communicatively couple to a service gateway 436 contained in data plane VCN 418 and one or more DB subnets 430 contained in data plane data layer 450. Data plane data layer 450 may include one or more DB subnets 430 that may communicatively couple to a service gateway 436 contained in data plane VCN 418.

[0078] One or more untrusted application subnets 462 may include a primary VNIC 464(1)-(N) communicatively coupled to tenant virtual machines (VMs) 466(1)-(N) residing within one or more untrusted application subnets 462. Each tenant VM 466(1)-(N) may run code in a corresponding container 467(1)-(N) and communicatively coupled to an application subnet 426 that may be contained in a data plane application layer 446, which may be contained in a container egress VCN 468. A corresponding secondary VNIC 472(1)-(N) may facilitate communication between one or more untrusted application subnets 462 contained in a data plane VCN 418 and the application subnets contained in a container egress VCN 468. The container egress VCN may include a public internet 454 (e.g., Figure 1 The public internet (154) uses NAT gateway 438.

[0079] Internet gateway 434, contained in control plane VCN 416 and data plane VCN 418, can be communicatively coupled to metadata management service 452 (e.g., Figure 1Metadata management service 452 can be communicatively coupled to the public internet 454. The public internet 454 can be communicatively coupled to a NAT gateway 438 contained in a control plane VCN 416 and a data plane VCN 418. A service gateway 436 contained in a control plane VCN 416 and a data plane VCN 418 can be communicatively coupled to a cloud service 456.

[0080] In some examples, Figure 4 The architecture shown in block diagram 400 can be considered as Figure 3 This is an exception to the pattern shown in the architecture of block diagram 300, and this pattern may be what the IaaS provider's customers would expect if the IaaS provider cannot communicate directly with the customer (e.g., in a disconnected region). The customer can have real-time access to the corresponding container 467(1)-(N) contained in each customer's VM 466(1)-(N). Container 467(1)-(N) can be configured to make calls to the corresponding auxiliary VNIC 472(1)-(N) contained in one or more application subnets 426 of the data plane application layer 446, which may be contained in the container egress VCN 468. The auxiliary VNIC 472(1)-(N) can transmit the calls to a NAT gateway 438, which can transmit the calls to the public internet 454. In this example, the container 467(1)-(N) that can be accessed by the customer in real-time can be isolated from the control plane VCN 416 and from other entities contained in the data plane VCN 418. Container 467(1)-(N) can also be isolated from resources from other customers.

[0081] In other examples, a client may use containers 467(1)-(N) to invoke cloud service 456. In this example, the client may run code within containers 467(1)-(N) requesting services from cloud service 456. Container 467(1)-(N) may transmit the request to a secondary VNIC 472(1)-(N), which may then transmit the request to a NAT gateway, which may transmit the request to the public internet 454. The public internet 454 may then transmit the request via internet gateway 434 to one or more LB subnets 422 contained in control plane VCN 416. In response to determining that the request is valid, one or more LB subnets may transmit the request to one or more application subnets 426, which may then transmit the request to cloud service 456 via service gateway 436.

[0082] It should be recognized that the IaaS architectures 100, 200, 300, and 400 depicted in the figures may have other components besides those depicted. Furthermore, the embodiments shown in the figures are merely some examples of cloud infrastructure systems that can be incorporated into embodiments of this disclosure. In some other embodiments, the IaaS system may have more or fewer components than shown in the figures, may combine two or more components, or may have different configurations or component arrangements.

[0083] In some embodiments, the IaaS system described herein may include application suites, middleware, and database service offerings delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such an IaaS system is the Oracle Cloud Infrastructure (OCI) provided by this assignee.

[0084] In one or more embodiments, a computer network provides connectivity between a set of nodes. Nodes may be local to each other and / or remote from each other. Nodes are connected via a set of links. Examples of links include coaxial cable, unshielded stranded cable, copper cable, fiber optic cable, and virtual links.

[0085] A subset of nodes implements a computer network. Examples of such nodes include switches, routers, firewalls, and Network Address Translation (NAT). Another subset of nodes uses a computer network. Such nodes (also called "hosts") can execute client processes and / or server processes. Client processes make requests for computing services, such as the execution of a specific application and / or the storage of a specific amount of data. Server processes respond by performing the requested service and / or returning the corresponding data.

[0086] A computer network can be a physical network, including physical nodes connected by physical links. A physical node is any digital device. A physical node can be a function-specific hardware device, such as a hardware switch, hardware router, hardware firewall, and hardware NAT. Additionally or alternatively, a physical node can be a general-purpose machine configured to perform various virtual machines and / or applications that implement corresponding functions. A physical link is the physical medium connecting two or more physical nodes. Examples of links include coaxial cable, unshielded stranded cable, copper cable, and fiber optic cable.

[0087] Computer networks can be overlay networks. An overlay network is a logical network implemented on top of another network, such as a physical network. Each node in an overlay network corresponds to a corresponding node in the underlying network. Therefore, each node in an overlay network is associated with both an overlay address (used to address the overlay node) and an underlying address (used to address the underlying node that implements the overlay node). Overlay nodes can be digital devices and / or software processes (such as virtual machines, application instances, or threads). The links connecting overlay nodes are implemented as tunnels through the underlying network. Overlay nodes at either end of the tunnel treat the underlying multi-hop path between them as a single logical link. Tunneling is performed through encapsulation and decapsulation.

[0088] In this embodiment, the client may be located locally and / or remotely on the computer network. The client may access the computer network via other computer networks, such as a private network or the Internet. The client may use a communication protocol, such as Hypertext Transfer Protocol (HTTP), to transmit requests to the computer network. Requests may be transmitted through interfaces such as client interfaces (e.g., web browsers), program interfaces, or APIs (Application Programming Interfaces).

[0089] In this embodiment, a computer network provides connectivity between clients and network resources. Network resources include hardware and / or software configured to execute server processes. Examples of network resources include processors, data storage, virtual machines, containers, and / or software applications. Network resources are shared among multiple clients. Clients independently request computing services from the computer network. Network resources are dynamically allocated to requesting and / or clients on demand. The network resources allocated to each requesting and / or client can be scaled up or down based on, for example, (a) computing services requested by a specific client, (b) aggregated computing services requested by a specific tenant, and / or (c) the requested aggregated computing services of the computer network. Such a computer network may be referred to as a "cloud network."

[0090] In this embodiment, the service provider offers a cloud network to one or more end users. The cloud network can implement various service models, including but not limited to Software as a Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service (IaaS). In SaaS, the service provider offers end users the ability to use applications running on the service provider's network resources. In PaaS, the service provider offers end users the ability to deploy custom applications onto network resources. Custom applications can be created using programming languages, libraries, services, and tools supported by the service provider. In IaaS, the service provider offers end users the ability to supply processing, storage, networking, and other basic computing resources provided by the network resources. Any application, including operating systems, can be deployed on the network resources.

[0091] In embodiments, computer networks can implement various deployment models, including but not limited to private clouds, public clouds, and hybrid clouds. In a private cloud, network resources are provisioned to a specific group of one or more entities (as used herein, an "entity" refers to a company, organization, person, or other entity) for exclusive use. Network resources can be local and / or remote at the location of the specific group of entities. In a public cloud, cloud resources are provisioned to multiple entities (also referred to as "tenants" or "customers") that are independent of each other. The computer network and its network resources are accessed by clients corresponding to different tenants. Such a computer network can be referred to as a "multi-tenant computer network." Several tenants can use the same specific network resources at different times and / or at the same time. Network resources can be local and / or remote at the tenant's location. In a hybrid cloud, the computer network includes both private and public clouds. The interface between the private and public clouds allows for the portability of data and applications. Data stored in the private cloud and data stored in the public cloud can be exchanged through the interface. Applications implemented in the private cloud and applications implemented in the public cloud may be dependent on each other. You can use the interface to make calls from an application in a private cloud to an application in a public cloud (and vice versa).

[0092] In this embodiment, the tenants of a multi-tenant computer network are independent of each other. For example, one tenant's business or operations may be separate from those of another tenant. Different tenants may have different network requirements for the computer network. Examples of network requirements include processing speed, data storage capacity, security requirements, performance requirements, throughput requirements, latency requirements, resilience requirements, quality of service (QoS) requirements, tenant isolation, and / or consistency. The same computer network may need to meet the different network requirements needed by different tenants.

[0093] In one or more embodiments, in a multi-tenant computer network, tenant isolation is implemented to ensure that applications and / or data from different tenants are not shared with each other. Various tenant isolation methods can be used.

[0094] In this embodiment, each tenant is associated with a tenant ID. Each network resource in a multi-tenant computer network is tagged with a tenant ID. A tenant is only allowed access to a specific network resource if the tenant and the specific network resource are associated with the same tenant ID.

[0095] In this embodiment, each tenant is associated with a tenant ID. Each application implemented by the computer network is tagged with a tenant ID. Additionally or alternatively, each data structure and / or dataset stored by the computer network is tagged with a tenant ID. A tenant is allowed access to a specific application, data structure, and / or dataset only if the tenant and the specific application, data structure, and / or dataset are associated with the same tenant ID.

[0096] As an example, each database implemented in a multi-tenant computer network can be identified by a tenant ID. Only the tenant associated with the corresponding tenant ID can access the data in a specific database. As another example, each entry in a database implemented in a multi-tenant computer network can be identified by a tenant ID. Only the tenant associated with the corresponding tenant ID can access the data in a specific entry. However, the database can be shared by multiple tenants.

[0097] In this embodiment, the subscription list indicates which tenants are authorized to access which applications. For each application, a list of tenant IDs of tenants authorized to access that application is stored. A tenant is only allowed to access a specific application if its tenant ID is included in the subscription list corresponding to that specific application.

[0098] In this embodiment, network resources (such as digital devices, virtual machines, application instances, and threads) corresponding to different tenants are isolated to tenant-specific overlay networks maintained by a multi-tenant computer network. As an example, data packets from any source device within a tenant overlay network may only be transmitted to other devices within the same tenant overlay network. Encapsulation tunnels are used to prevent any transmission from a source device on one tenant overlay network to devices in other tenant overlay networks. Specifically, data packets received from the source device are encapsulated within an outer data packet. The outer data packet is transmitted from a first encapsulation tunnel endpoint (communicating with the source device in the tenant overlay network) to a second encapsulation tunnel endpoint (communicating with the destination device in the tenant overlay network). The second encapsulation tunnel endpoint decapsulates the outer data packet to obtain the original data packet transmitted by the source device. The original data packet is transmitted from the second encapsulation tunnel endpoint to the destination device within the same specific overlay network.

[0099] 4. Computer System

[0100] Figure 5 An example computer system 500 in which various embodiments can be implemented is illustrated. System 500 can be used to implement any of the computer systems described above. Figure 5 As shown, the computer system 500 includes a processing unit 504 that communicates with multiple peripheral subsystems via a bus subsystem 502. These peripheral subsystems may include a processing acceleration unit 506, an I / O subsystem 508, a storage subsystem 518, and a communication subsystem 524. The storage subsystem 518 includes a tangible computer-readable storage medium 522 and a system memory 510.

[0101] Bus subsystem 502 provides a mechanism for allowing various components and subsystems of computer system 500 to communicate with each other as intended. While bus subsystem 502 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 502 can be any of several types of bus architectures, including memory buses or memory controllers, peripheral buses, and local buses using any of the various bus architectures available. For example, such architectures may include Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses, which may be implemented as Mezzanine buses manufactured according to the IEEE P1386.1 standard.

[0102] A processing unit 504, which may be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of the computer system 500. One or more processors may be included in the processing unit 504. These processors may include single-core or multi-core processors. In some embodiments, the processing unit 504 may be implemented as one or more independent processing units 532 and / or 534, wherein each processing unit includes a single-core or multi-core processor. In other embodiments, the processing unit 504 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0103] In various embodiments, processing unit 504 can execute various programs in response to program code and can maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed can reside in processing unit 504 and / or storage subsystem 518. With appropriate programming, processing unit 504 can provide the various functions described above. Computer system 500 may additionally include processing acceleration unit 506, which may include digital signal processor (DSP), dedicated processor, etc.

[0104] I / O subsystem 508 may include user interface input devices and user interface output devices. User interface input devices may include keyboards, pointing devices such as mice or trackballs, touchpads or touchscreens integrated into a display, scroll wheels, click wheels, dials, buttons, switches, keyboards, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may include, for example, motion sensing and / or gesture recognition devices, such as the Microsoft Kinect® motion sensor, which enables users to control and interact with input devices such as the Microsoft Xbox® 360 game controller via a natural user interface using gestures and voice commands. User interface input devices may also include eye posture recognition devices, such as the Google Glass® blink detector, which detects eye activity from the user (e.g., “blinking” when taking a photo and / or making menu selections) and translates the eye posture into input in an input device (e.g., Google Glass®). Furthermore, user interface input devices may include voice recognition sensing devices that enable users to interact with a voice recognition system (e.g., the Siri® navigator) via voice commands.

[0105] User interface input devices may also include, but are not limited to, 3D mice, joysticks or pointing sticks, game panels and drawing tablets, as well as audio / video devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. Furthermore, user interface input devices may include, for example, medical imaging input devices such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and medical ultrasound equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards, digital musical instruments, etc.

[0106] User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices, etc. Display subsystems may be cathode ray tubes (CRTs), flat panel devices such as those using liquid crystal displays (LCDs) or plasma displays, projection devices, touchscreens, etc. Generally, the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from computer system 500 to a user or other computer. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0107] Computer system 500 may include a storage subsystem 518 that provides a tangible, non-transitory, computer-readable storage medium for storing software and data constructs that provide the functionality of the embodiments described in this disclosure. The software may include programs, code modules, instructions, scripts, etc., which provide the aforementioned functionality when executed by one or more cores or processors of processing unit 504. Storage subsystem 518 may also provide a repository for storing data used according to this disclosure.

[0108] like Figure 5 As depicted in the example, storage subsystem 518 may include various components, including system memory 510, computer-readable storage medium 522, and computer-readable storage medium reader 520. System memory 510 may store program instructions, such as application program 512, that can be loaded and executed by processing unit 504. System memory 510 may also store data used during the execution of instructions (such as program data 514) and / or data generated during the execution of program instructions. Various kinds of programs may be loaded into system memory 510, including but not limited to client applications, web browsers, middleware applications, relational database management systems (RDBMS), virtual machines, containers, etc.

[0109] System memory 510 may also store operating system 516. Examples of operating system 516 may include various versions of Microsoft Windows®, Apple Macintosh® and / or Linux operating systems, various commercial UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.) and / or mobile operating systems (such as iOS, Windows® Phone, Android® OS, BlackBerry® OS, and Palm® OS). In some implementations where computer system 500 executes one or more virtual machines, the virtual machine, along with its guest operating system (GOS), may be loaded into system memory 510 and executed by one or more processors or cores of processing unit 504.

[0110] System memory 510 may be configured differently depending on the type of computer system 500. For example, system memory 510 may be volatile memory (such as random access memory (RAM)) and / or non-volatile memory (such as read-only memory (ROM), flash memory, etc.). Different types of RAM configurations may be provided, including static random access memory (SRAM), dynamic random access memory (DRAM), etc. In some embodiments, system memory 510 may include a basic input / output system (BIOS) containing basic routines that facilitate, for example, the transfer of information between components within the computer system 500 during startup.

[0111] Computer-readable storage medium 522 may represent remote, local, fixed and / or removable storage devices and storage media for temporarily and / or more permanently containing and storing computer-readable information (including instructions executable by the processing unit 504 of the computer system 500) for use by the computer system 500.

[0112] Computer-readable storage medium 522 may include any suitable medium known or used in the art, including storage and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing and / or transmitting information. This may include tangible computer-readable storage media such as RAM, ROM, electrically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassette, magnetic tape, disk storage or other magnetic storage devices, or other tangible computer-readable media.

[0113] For example, computer-readable storage medium 522 may include a hard disk drive that reads or writes to non-removable non-volatile magnetic media, a disk drive that reads or writes to removable non-volatile disks, and an optical disc drive that reads or writes to removable non-volatile optical discs (such as CD-ROMs, DVDs, and Blu-ray® discs or other optical media). Computer-readable storage medium 522 may include, but is not limited to, Zip® drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVD discs, digital audio tapes, and so on. Computer-readable storage medium 522 may also include solid-state drives (SSDs) based on non-volatile memory (such as flash memory-based SSDs, enterprise flash drives, solid-state ROMs, etc.), volatile memory-based SSDs (such as SSDs based on solid-state RAM, dynamic RAM, static RAM, DRAM, magnetoresistive RAM (MRAM) SSDs), and hybrid SSDs using a combination of DRAM-based and flash memory-based SSDs. Disk drives and their associated computer-readable media can provide non-volatile storage for computer-readable instructions, data structures, program modules and other data for computer system 500.

[0114] Machine-readable instructions executable by one or more processors or cores of processing unit 504 may be stored on a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium may include physically tangible memory or storage devices, including volatile memory storage devices and / or non-volatile memory devices. Examples of non-transitory computer-readable storage media include magnetic storage media (e.g., disks or tapes), optical storage media (e.g., DVDs, CDs), various types of RAM, ROM, or flash memory, hard disk drives, floppy disk drives, removable memory drives (e.g., USB drives), or other types of storage devices.

[0115] The communication subsystem 524 provides interfaces to other computer systems and networks. The communication subsystem 524 acts as an interface for receiving data from other systems and sending data from computer system 500 to other systems. For example, the communication subsystem 524 enables computer system 500 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 524 may include radio frequency (RF) transceiver components (e.g., advanced data network technologies using cellular telephone technologies, such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 802.11 series standards), or other mobile communication technologies, or any combination thereof), global positioning system (GPS) receiver components, and / or other components for accessing wireless voice and / or data networks. In some embodiments, as an addition to or alternative to the wireless interface, the communication subsystem 524 may provide a wired network connection (e.g., Ethernet).

[0116] In some embodiments, the communication subsystem 524 may also represent one or more users who can use the computer system 500 to receive input communications in the form of structured and / or unstructured data feeds 526, event streams 528, event updates 530, etc.

[0117] For example, the communication subsystem 524 can be configured to receive data feeds 526 in real time from users of social networks and / or other communication services, such as Twitter® feeds, Facebook® updates, web feeds such as Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0118] Furthermore, the communication subsystem 524 can also be configured to receive data in the form of a continuous data stream, which may include an event stream 528 and / or event updates 530 that are essentially continuous or unbounded real-time events without a clearly defined termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial quote machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, and so on.

[0119] The communication subsystem 524 can also be configured to output structured and / or unstructured data feeds 526, event streams 528, event updates 530, etc. to one or more databases, which can communicate with one or more streaming data source computers coupled to the computer system 500.

[0120] The computer system 500 can be one of a variety of types, including handheld portable devices (e.g., iPhone® cellular phones, iPad® computing tablets, PDAs), wearable devices (e.g., Google Glass® head-mounted displays), PCs, workstations, mainframes, information stations, server racks, or any other data processing system.

[0121] Due to the ever-changing nature of computers and networks, Figure 5 The description of computer system 500 shown in the image is merely a concrete example. It has more than Figure 5 Many other configurations of the system, with more or fewer components, are possible. For example, custom hardware may be used and / or specific elements may be implemented using hardware, firmware, software (including applets), or a combination thereof. Additionally, connections to other computing devices, such as network input / output devices, may also be employed. Based on the disclosure and teachings provided herein, those skilled in the art will recognize other ways and / or methods for implementing the various embodiments.

[0122] 5. System Architecture Overview

[0123] Figure 6A and Figure 6B An example of a system 600 according to one or more embodiments is illustrated. References Figure 6A and Figure 6B The described system 600 can be used to distribute certificate bundles to network entities associated with a virtual cloud network. For example... Figure 6A and Figure 6B As shown, system 600 may include at least one distribution service 602 and a group of network entities 604. Distribution service 602 may be configured to distribute certificate bundles, including CA certificates, to the group of network entities.

[0124] Certificate bundles can be housed in certificate store 606. Distribution service 602 can retrieve certificate bundles from certificate store 606 and send them to the group of network entities 604. In one example, certificate store 606 is any type of storage unit and / or device used for storing data (e.g., file system, database, table collection, or any other storage facility). Alternatively, certificate store 606 may include multiple different storage units and / or devices. These multiple different storage units and / or devices may or may not be of the same type or located at the same physical site. Furthermore, certificate store 606 may be implemented or executed on the same computing system as distribution service 602 and / or the group of network entities 604. Additionally, or alternatively, certificate store 606 may be implemented or executed on a separate computing system from distribution service 602 and / or the group of network entities 604. Certificate store 606 may be coupled to distribution service 602 via a direct connection or via network communication.

[0125] In one example, such as Figure 6A As shown, system 600 may include distribution service 602a and distribution service 602n. Distribution service 602a may distribute certificate bundles to a group of network entities 604, including network entity 604a and network entity 604n. Additionally, or alternatively, distribution service 602n may distribute certificate bundles to another group of network entities 604, including network entity 604x and network entity 604z.

[0126] Network entity 604 may include update module 608 and storage medium 610. In one example, network entity 604a may include update module 608a and storage medium 610a. Additionally, or alternatively, network entity 604n may include update module 608n and storage medium 610n. Update module 608 may periodically request updated certificate bundles from distribution service 602. In response to a request for an updated certificate bundle from update module 608, distribution service 602 may distribute the certificate bundle to update module 602. Additionally, or alternatively, distribution service may push the updated certificate bundle to update module 608, for example, if no request is received from update module 608.

[0127] In one example, distribution service 602 may distribute certificate bundles to a set of network entities 604 associated with a region. For example, distribution service 602a may distribute certificate bundles to network entities 604a and 604n associated with region 612a. Additionally, or alternatively, distribution service 602n may distribute certificate bundles to network entities 604x and 604z associated with region 612n. In one example, system 600 may include a set of regions 612, such as regions 612a and 612n. In one example, this set of regions may be associated with domain 614. In one example, system 600 may include a set of domains 614, each of which includes a set of regions 612. As used herein, the term "region" refers to a group of one or more interconnected data centers on which a virtual cloud network is deployed. In one example, a region may correspond to a physical geographic area. In one example, a region may correspond to multiple physical geographic areas and / or a portion of physical geographic areas. As used herein, the term "domain" refers to a unique deployment of IaaS system infrastructure. In one example, regions within a domain may communicate with each other. In one example, regions from different areas may not be able to communicate with each other.

[0128] Network entity 604 may include endpoints associated with a virtual cloud network, such as nodes, hosts, agents, services, components, or other elements, implemented in hardware and / or software. Network entity 604 may reside within the base layer, the overlay instance, or within the interface between the base layer and the overlay instance.

[0129] In one example, the group of network entities 604 may include a set of base layer entities. As used herein, the term "base layer entity" refers to network entity 604 implemented in the base layer network. As used herein, the term "base layer network" refers to physical network infrastructure. A base layer network generally provides the foundation for a virtual cloud network. A base layer network may include physical network devices such as routers, switches, network links, and other networking components. A base layer network generally provides the basic connectivity and transport capabilities necessary for data transmission within and between data centers.

[0130] A set of base-level entities may include base-level hosts (e.g., bare-metal hosts), routers, firewall devices, load balancers, storage devices, and / or base-level services. Base-level services may include one or more services that execute or are executable on the base-level entities, such as firmware services, network connectivity services, addressing services, name resolution services, security services, network monitoring services, load balancing services, and / or storage services. Firmware services may be associated with the functionality or management of network infrastructure components or services, such as network devices, startup or initialization processes, hardware control, feature enabling, updates, hardware abstraction, network configuration, and / or network management. In one example, a base-level entity may include a combination of hardware and software.

[0131] In one example, the group of network entities 604 may include a set of overlay entities. As used herein, the term "overlay entity" refers to network entity 604 implemented within an overlay network. As used herein, the term "overlay network" refers to a virtual network built on an underlying network using software-defined networking (SDN), virtualization, tunneling, and / or encapsulation technologies. Overlay networks generally operate independently of the underlying underlying network. Overlay networks can provide logical separation and isolation of traffic, enable virtual network provisioning, and / or allow the implementation of various network services and policies. Virtual machines, hosts, containers, or virtual network functions running on the underlying network can be connected via the overlay network.

[0132] A set of overlay entities may include overlay hosts, subnets, overlay controllers, and / or overlay clients. Overlay hosts may include endpoints within the overlay network, such as virtual machines, containers, or physical servers. Overlay hosts can communicate with each other using logical network addresses assigned within the overlay network. Subnets may include virtual network segments with unique addressing schemes and / or a unique set of network policies and / or services. A subnet may include a set of overlay hosts. Multiple subnets may be used to partition corresponding sets of overlay hosts. An overlay controller may oversee the management, control, provisioning, configuration, and / or monitoring of the overlay network, network entities on the overlay network, and / or network policies within the overlay. The overlay controller interacts with the underlying layer network, for example, to coordinate the operation of overlay hosts and / or communication across virtual switches and tunnels. Overlay clients may include endpoints or devices that initiate communication within the overlay network. Overlay clients may be specific instances or roles within an overlay host. An overlay host may include a set of overlay clients. Overlay clients may include consumers or users of services provided by the overlay host or IaaS. Overlay clients may request and consume resources or services from the overlay host, thus acting as consumers or clients of those resources or services. Overlay services may include one or more services that are executed or executable on the overlay entity. Overlay services may include client-specific services, such as services installed by the client. Additionally, or alternatively, overlay services may include virtual network creation services, virtual network management services, virtual machine orchestration services, container orchestration services, network virtualization services, overlay security services, load balancing services, multi-tenancy services, and / or tenant isolation services.

[0133] In one example, the group of network entities 604 may include a group of interface entities. As used herein, the term "interface entity" refers to network entity 604 implemented on a network interface. As used herein, the term "network interface" refers to a communication interface between the overlay network and the underlying network, such as a network interface card, a smartNIC, etc. An interface entity may include a network interface card, such as a smartNIC, a node on a network interface card, or an interface service that is performed or executable on the interface entity. A node on a network interface card may include a programmable hardware component, a memory component, or a gateway component.

[0134] Gateway components can provide connectivity between overlay networks and base networks. For example, a gateway component can support communication between overlay entities and base entities. Additionally, or alternatively, gateway components can provide connectivity between overlay networks and external networks, such as the Internet or other networks outside the overlay network. For example, an overlay gateway can support communication between overlay entities and external endpoints.

[0135] Further reference Figure 6BIn one example, system 600 may include a base network 620, an overlay network 622, and an interface entity 624 providing communication between the base network 620 and the overlay network 622. The base network 620 may include a certificate store 626, a distribution service 602, and one or more base entities 628. These one or more base entities 628 may include one or more base hosts 632 (e.g., base host 632a and base host 632n), and / or one or more base services 634 (e.g., base service 632a and base service 632n). In one example, base host 632 may include a bare-metal host. In one example, base service 634 may include a firmware service. Overlay network 622 may include a set of overlay entities 630, such as overlay entity 630a and overlay entity 630n. In one example, this set of overlay entities 630 may include overlay hosts. Additionally, or alternatively, this set of overlay entities 630 may include overlay services. In one example, interface entity 624 may include a network interface card, such as a smartNIC. Additionally, or alternatively, interface entity 624 may include nodes or endpoints on a network interface card or smartNIC.

[0136] like Figure 6B As shown, system 600 may include distribution service 602. Distribution service 602 may be implemented on base network 620. Distribution service 602 may be communicatively coupled to one or more base entities 628 and / or one or more interface entities 624. Distribution service 602 may obtain certificate bundles from certificate store 606. Certificate store 606 may include storage media on base network 620 or other physical hardware of virtual cloud network. In one example, distribution service 602 may distribute certificate bundles to the group of base entities 628, such as to the group of base hosts 632 and / or the group of base services 634. Additionally, or alternatively, distribution service 602 may distribute certificate bundles to interface entities 624.

[0137] For example, further Figure 6B As shown, interface entity 624 may include forwarding module 636. Forwarding module 636 may forward certificate bundles (such as certificate bundles received by interface entity 624 from distribution service 602) to a set of overlay entities 630 on overlay network 622. In response to a request for an updated certificate bundle from overlay entity 630, forwarding module 636 may forward the certificate bundle to overlay entity 630. Additionally, or alternatively, forwarding module 636 may push the updated certificate bundle to overlay entity 630, for example, if no request is received from overlay entity 630.

[0138] Requests for updated certificate bundles from the group of overriding entities 603 can be provided through interface 638 between the interface entity and the group of overriding entities 630. Alternatively, the certificate bundle can be forwarded to the group of overriding entities 630 via interface 638. In one example, interface 638 can be a metadata interface configured to exchange metadata between interface entity 624 and the group of overriding entities 630. In one example, interface 638 can be utilized between interface entity 624 and each of the overriding entities 630 in the group. Alternatively, interface entity 624 can have a set of interfaces 638, each interface 638 corresponding to a corresponding overriding entity 630.

[0139] like Figure 6B As shown, and further referenced Figure 6A Each network entity 604 may include a storage medium 610 and an update module 608 associated with the network entity 604. In one example, regarding the base entity 628, the distribution service 602 may distribute a certificate bundle to the update module 608b of the base host 632a, and the certificate bundle may be stored in the storage medium 610b. Additionally, or alternatively, the distribution service 602 may distribute a certificate bundle to the update module 608c of the base service 634a, and the certificate bundle may be stored in the storage medium 610c.

[0140] In one example, regarding interface entity 624, distribution service 602 can distribute a certificate bundle to update module 608d, and the certificate bundle can be stored in storage medium 610d. Forwarding module 636 can obtain the certificate bundle from storage medium 610d and forward the certificate bundle to the overlay entity 630.

[0141] In one example, regarding the set of overlay entities 630, the forwarding module 636 can forward certificate bundles to the update module 608e of the overlay entity 630a, and the certificate bundles can be stored in the storage medium 610e. Additionally, or alternatively, the forwarding module 636 can forward certificate bundles to the update module 608f of the overlay entity 630n, and the certificate bundles can be stored in the storage medium 610f.

[0142] In one or more embodiments, system 600 may include a ratio Figure 6A or Figure 6B The components shown may have more or fewer components. Figure 6A and Figure 6B The components shown can be located locally to each other or far from each other. Figure 6A and Figure 6BThe components shown may include software and / or hardware components. Each component may be distributed across multiple applications and / or machines. Multiple components may be combined into one application and / or machine. An operation described for one component may be performed alternatively by another component. Additional embodiments and / or examples related to system 600 are described in Section 2 above, entitled “Cloud Computing Technology”.

[0143] In an embodiment, system 600 may include various components implemented on one or more digital devices. The term "digital device" generally refers to any hardware device that includes a processor. A digital device may refer to a physical device that executes an application or a virtual machine. Examples of digital devices include computers, tablets, laptops, desktops, netbooks, servers, web servers, network policy servers, proxy servers, general-purpose machines, function-specific hardware devices, hardware routers, hardware switches, hardware firewalls, hardware network address translation (NAT), hardware load balancers, mainframes, televisions, content receivers, set-top boxes, printers, mobile handsets, smartphones, personal digital assistants (PDAs), wireless receivers and / or transmitters, base stations, communication management equipment, routers, switches, controllers, access points, and / or client devices.

[0144] 6. Authenticating network entities

[0145] Network entities can use CA certificates from the certificate bundle to authenticate other network entities associated with the virtual cloud network. For example, communication between network entities can be conducted according to a security protocol. This security protocol may include authenticating network entities based on entity certificates issued by the CA, for example, before establishing communication with a network entity.

[0146] In one example, an entity certificate and the CA certificate corresponding to the CA that issued the entity certificate can represent at least a portion of a certificate chain. Each signature-key pair in the certificate chain is verified for the purpose of authenticating a network entity. In one example, a top-level CA can issue entity certificates; in this case, the certificate chain can include a single signature-key pair: the digital signature of the top-level CA in the entity certificate and the public key of the top-level CA. Such a top-level CA is sometimes referred to as the root CA. In another example, a certificate chain can include signature-key pairs corresponding to multiple CA certificates. For example, a root CA can issue intermediate CA certificates to intermediate CAs, and the intermediate CAs can issue entity certificates to network entities. In this case, the certificate chain includes two signature-key pairs: (i) the digital signature of the intermediate CA in the entity certificate and the public key of the intermediate CA; and (ii) the digital signature of the root CA in the intermediate CA certificate and the public key of the root CA.

[0147] As used herein, the terms "Certificate Authority Certificate" or "CA Certificate" refer to a digital certificate issued by a Certificate Authority (CA) to establish its own identity and authenticity. A Certificate Authority Certificate can be a root CA certificate or an intermediate CA certificate. Certificate Authority Certificates can be used to sign and issue other digital certificates, including those for secure communication between network entities.

[0148] As used herein, the term "Certificate Authority" or "CA" refers to the entity responsible for issuing and managing digital certificates. A CA verifies the identity of a network entity and digitally signs its certificates to prove their authenticity.

[0149] As used herein, the term "root certificate authority certificate" or "root CA certificate" refers to the top-level CA certificate in a certificate chain or hierarchy. A root CA certificate can be self-issued and / or self-signed by the root CA. As used herein, the term "root CA" refers to the top-level CA in a CA hierarchy. A root CA can issue root CA certificates, intermediate CA certificates, or entity certificates.

[0150] As used herein, the term "intermediate certificate authority certificate" or "intermediate CA certificate" refers to an intermediate-level CA certificate in a certificate chain or hierarchy. An intermediate CA certificate can be issued by a root CA. An intermediate CA certificate sits between the root CA certificate and the entity certificate in a certificate chain or hierarchy. As used herein, the term "intermediate CA" refers to an intermediate-level CA in a CA hierarchy. For example, an intermediate CA can issue entity certificates, for instance, according to the permissions granted to the intermediate CA by the root CA.

[0151] As used herein, the term "entity certificate" refers to a digital certificate issued to an entity, such as a network entity associated with a virtual cloud network. Entity certificates can be used to verify the identity of an entity and support secure communication between entities, such as between network entities within a virtual cloud network. Entity certificates can be issued by a CA (such as a root CA or an intermediate CA).

[0152] In one example, the entity certificate could be an instance master certificate. As used herein, the term "instance master certificate" refers to a digital certificate used to authenticate and secure communications for an instance or VM associated with a virtual cloud network. In one example, instances and VMs can be dynamically created, extended, and terminated. Instance master certificates can be associated with an instance or VAM for its lifetime and can be automatically generated and managed by the virtual cloud network infrastructure. Instance master certificates can have limited access rights to communicate with certain network entities based on permissions assigned to the network entity that issued the instance master certificate.

[0153] As used herein, the term "digital certificate" refers to a digitally signed electronic document that binds a public key to the identity of an entity. Digital certificates may conform to the International Telecommunication Union standard X.509. A digital certificate may include the name of the issuer, the name of the certificate holder, the public key, the issuer (CA) information, and the expiration date. Digital certificates can be used in various security protocols, such as SSL / TLS, to establish the identity and authenticity of communicating parties and facilitate secure communication.

[0154] 6. Update Certificate Authority Certificates

[0155] Now for reference Figure 7 The following describes further operations involving updating the CA certificate. Figure 7 One or more operations 700 shown in the diagram can be modified, rearranged, or omitted entirely. Accordingly, Figure 7 The specific sequence of operations 700 illustrated in the diagram should not be construed as limiting the scope of one or more embodiments. Figure 7 As shown, operation 700 for distributing the certificate bundle may include a series or sequence of stages. In one example, as shown, operation 700 may include a series or sequence of stages, such as four (4) stages. In another example, this series or sequence of stages may include more or fewer than four (4) stages, including Figure 7 All or part of the stages shown.

[0156] Figure 7 The illustration depicts a set of example operations 700 associated with updating a CA certificate according to one or more embodiments. The CA certificate may include a root CA certificate and / or intermediate CA certificates. The CA certificate may be contained in a certificate bundle. In one example, operation 700 may include, at block 702, distributing an intermediate certificate bundle containing a set of CA certificates to a set of network entities associated with a virtual cloud network. Additionally, or alternatively, a new set of CA certificates may be added to an existing certificate bundle. The operation at block 702 may represent phase 1 of the certificate bundle distribution process.

[0157] In one example, operation 700 may include, at box 704, activating the CA certificate in the intermediate certificate bundle. In one example, activating the CA certificate may include issuing one or more intermediate CA certificates to the CA corresponding to the CA certificate. The operation at box 704 may represent phase 2 of the certificate bundle distribution process.

[0158] In one example, operation 700 may include, at block 706, distributing the updated entity certificate to the group of network entities. Additionally, or alternatively, operation 700 may include distributing an updated intermediate CA certificate, for example, corresponding to the updated entity certificate. The operation at block 706 may represent phase 3 of the certificate bundle distribution process.

[0159] In one example, operation 700 may include, at box 708, removing an older CA certificate. Removing an older CA certificate may include distributing an updated certificate bundle containing only the new CA certificate, thereby effectively removing the old CA certificate. Additionally, or alternatively, removing an older CA certificate may include instructing a network entity to remove the old CA certificate from the updated certificate bundle. The operation at box 708 may represent stage 4 of the certificate bundle distribution process.

[0160] 8. Distribute the certificate bundle

[0161] A. Distribution Operation Timeline

[0162] Now for reference Figures 8A to 8D This further describes the operations involved in distributing certificate bundles to network entities associated with the virtual cloud network. (See reference) Figures 8A to 8D One or more operations 800 described may be modified, rearranged, or omitted entirely. Accordingly, refer to... Figures 8A to 8D The specific sequence of operations 800 described should not be construed as limiting the scope of one or more embodiments. In one example, reference... Figures 8A to 8D The described operation 800 can be compared with the reference. Figure 7 The described operation corresponds to phase 1 and / or phase 4.

[0163] Figures 8A to 8D The illustration depicts a set of example operations 800 involving the distribution of a certificate bundle according to one or more embodiments. The certificate bundle may include a set of CA certificates, including a root CA certificate and / or intermediate CA certificates. Figures 8A to 8D The operation 800 shown represents a timeline from top to bottom. Operation 800 may include the operation of distribution service 802 shown on distribution service timeline 804. Additionally, or alternatively, operation 800 may include the operation of network entity 806 and / or update module 808 shown on network entity timeline 810. (See reference...) Figures 8A to 8D The described operation 800 can be performed on a set of network entities 806 associated with the virtual cloud network. Additionally, or alternatively, refer to... Figures 8A to 8D The described operation 800 can be performed on at least one distribution service 802 associated with a set of network entities 806.

[0164] refer to Figure 8AAt time 812, distribution service 802 can obtain an initial certificate bundle containing a set of current CA certificates. Distribution service 802 can distribute the certificate bundle to various network entities, and network entities can use the CA certificates in the certificate bundle to verify network entities associated with the virtual cloud network. For example... Figure 8A As shown, at time 814, network entity 806 may request a certificate bundle. For example, update module 808 of network entity 806 may send a request for the certificate bundle to distribution service 802. At time 816, distribution service 802 may receive the request for the certificate bundle. In response to the request, distribution service 802 may send the certificate bundle to network entity 806, such as to update module 808 of network entity 806. At time 818, update module 808 may install the certificate bundle received from distribution service 802.

[0165] In one example, update module 808 may periodically request certificate bundles from distribution service 802. Distribution service 802 may respond to the periodic request by sending a certificate bundle to update module 808. In one example, distribution service 802 may send a certificate bundle to update module 808, for example, in response to a periodic request from update module 808, when an updated certificate bundle is available. When an updated certificate bundle is unavailable, distribution service 802 may avoid sending a certificate bundle to update module 808 in response to a periodic request. In one example, when an updated certificate bundle is unavailable, such as when distribution service 802 has previously sent a latest certificate bundle to update module 808, update module 808 may respond to the periodic request from distribution service 802 with a message indicating that the updated certificate bundle is unavailable. In one example, the periodic request from update module 808 may include an indication of the version of the certificate bundle being used by network entity 806. Distribution service 802 can determine whether to send a certificate bundle to update module 808 based on the version indication included in the periodic request.

[0166] At time 820, distribution service 802 can update the initial certificate bundle with the updated certificate bundle. The updated certificate bundle can be an intermediate certificate bundle. Distribution service 802 can receive intermediate certificate bundles from a source, such as from a CA. Additionally, or alternatively, distribution service 802 can generate intermediate certificate bundles. In one example, distribution service 802 can receive one or more new CA certificates, such as from one or more CAs, and distribution service 802 can generate an intermediate certificate bundle that includes these one or more new CA certificates. For example, distribution service 802 can generate an intermediate certificate bundle by adding a new CA certificate to a previous certificate bundle that includes an earlier CA certificate. The new CA certificate can include a new CA certificate or a pre-existing CA certificate. For example, when a new CA certificate is intended to replace an earlier CA certificate, the new CA certificate can be newly issued and included in the intermediate certificate bundle. As another example, when a network entity intends to begin communication with a pre-existing network entity associated with a CA certificate, a pre-existing new CA certificate can be included in the intermediate certificate bundle.

[0167] At time 822, updated certificate bundles (such as intermediate certificate bundles) are available for distribution service 802. An intermediate certificate bundle may include a set of earlier CA certificates that were included in the certificate bundle prior to the update, and a set of new CA certificates added to the certificate bundle in conjunction with the update. The set of earlier CA certificates included in the intermediate certificate bundle may currently be used by network entities to authenticate other entities. The set of new CA certificates included in the intermediate certificate bundle may replace at least some of the earlier CA certificates. The set of CA certificates added to the certificate bundle in conjunction with the update may sometimes be referred to as the new CA certificates. At time 822, an intermediate certificate bundle may include a set of earlier CA certificates and a set of new CA certificates.

[0168] At time 824, coinciding with time 822, distribution service 802 may send an intermediate certificate bundle to network entity 806, such as to update module 808. In one example, distribution service 802 may push the intermediate certificate bundle to network entity 806 in response to the intermediate certificate bundle becoming available. Additionally, or alternatively, at time 826, update module 808 may send a request to distribution service 802, and distribution service 802 may send the intermediate certificate bundle to update module 808 in response to the request. Update module 808 may receive the intermediate certificate bundle in response to the request at time 826.

[0169] At time 828, update module 808 can verify the intermediate certificate bundle received from distribution service 802. In one example, this can be done according to reference... Figures 9A to 9COne or more operations described verify the intermediate certificate bundle. In response to the verification of the intermediate certificate bundle, update module 808 may install the intermediate certificate bundle at time 830. After the intermediate certificate bundle is verified and installed by update module 808, network entity 806 may begin using at least some of the CA certificates in the intermediate certificate bundle to authenticate other network entities associated with the virtual cloud network. For example, network entity 806 may use an earlier CA certificate from the intermediate certificate bundle to authenticate other network entities. In one example, a new CA certificate may need to be activated before it is used to authenticate network entities. In one example, a new CA certificate may be activated before being added to the intermediate certificate bundle and / or before the intermediate certificate bundle is distributed by distribution service 802.

[0170] Now for reference Figure 8B At time 832, the distribution service may generate additional updated certificate bundles, such as a second certificate bundle. The second certificate bundle may reflect an update to the intermediate certificate bundle distributed by the distribution service. An update from the intermediate certificate bundle to the second certificate bundle may include the removal of an earlier CA certificate. The second certificate bundle may be generated either by removing an earlier CA certificate from the intermediate certificate bundle or by creating a new certificate bundle that includes at least one of the new CA certificates but does not include the earlier CA certificates. One or more operations performed to remove the earlier CA certificates may sometimes be referred to as consolidating the certificate bundle. In one example, operations associated with consolidating the certificate bundle may include: verifying that the network entity has distributed the new CA certificate in the intermediate certificate bundle, activating the new CA certificate, and removing the earlier CA certificate. The earlier CA certificates removed from the certificate bundle may include one or more CA certificates that have been replaced by their corresponding new CA certificates. All or part of the earlier CA certificates may be replaced by their corresponding new CA certificates. In one example, the second certificate bundle may include one or more earlier CA certificates that have not been replaced by the new CA certificates. These one or more unreplaced CA certificates can remain valid after the certificate bundle is integrated.

[0171] In one example, at time 834, distribution service 802 may verify that network entity 806 has a new CA certificate. Distribution service 802 may perform the verification, or the verification may be based on the fact that an intermediate certificate bundle has been distributed to the network entity. Additionally, or alternatively, the verification may be based at least in part on a verification message from network entity 806 and / or the elapsed time if no error message is received from network entity 806.

[0172] At time 836, the distribution service 802 can activate a new CA certificate, for example, as referenced. Figure 7As described in box 704. Furthermore, at time 836, the distribution service 802 can distribute new entity certificates, for example, as referenced. Figure 7 As described in box 706. In one example, activating a new CA certificate may include issuing one or more intermediate CA certificates to a CA corresponding to one or more new root CA certificates. In one example, distributing a new entity certificate may include issuing one or more entity certificates to a corresponding one or more network entities to a CA corresponding to the one or more intermediate CA certificates. Additionally, or alternatively, a CA corresponding to the one or more new root CA certificates may issue one or more entity certificates to a corresponding one or more network entities. The new root CA certificate may be used by other network entities to authenticate entity certificates and / or intermediate CA certificates.

[0173] At time 838, the distribution service 802 may remove an earlier CA certificate. Removing an earlier CA certificate may include generating a second certificate bundle that does not include the earlier CA certificate. For example, a second certificate bundle may be defined, and a new CA certificate may be added to the second certificate bundle (along with any earlier CA certificates that were not replaced by the new CA certificate). Additionally, or alternatively, the earlier CA certificate may be removed from an intermediate certificate bundle that includes the new CA certificate, and this intermediate certificate bundle may subsequently be defined as the second certificate bundle. As a result of removing the earlier CA certificate, at time 840, the second certificate bundle may include the new CA certificate and any earlier CA certificates that were not replaced by the new CA certificate.

[0174] At time 842, coinciding with time 840, update module 808 may send a request for the updated certificate bundle to distribution service 802. The request from update module 808 may be one of a series of periodic requests. At time 844, coinciding with time 840, distribution service 802 may send the second certificate bundle to network entity 806, such as update module 808 sending it to network entity 806. Update module 808 may receive the second certificate bundle in response to this request at time 842. Additionally, or alternatively, distribution service 802 may push the second certificate bundle to update module 808 in response to the second certificate bundle becoming available.

[0175] In response to receiving the second certificate bundle, at time 846, update module 808 can verify the second certificate bundle received from distribution service 802. In one example, this can be done according to reference... Figures 9A to 9COne or more operations described verify the second certificate bundle. In response to the verification of the second certificate bundle, update module 808 may install the second certificate bundle at time 848. After update module 808 verifies and installs the second certificate bundle, network entity 806 may begin using the CA certificate in the second certificate bundle to authenticate other network entities associated with the virtual cloud network. For example, the new CA certificate may be used to authenticate network entities. Additionally, or alternatively, any earlier CA certificate included in the second certificate bundle (e.g., if not superseded by the new CA certificate) may be used to authenticate network entities.

[0176] Now for reference Figure 8C and Figure 8D Further, an example operation involving the distribution of certificate bundles is described in 800. Figure 8A and Figure 8D As shown, a certificate bundle distributed to a network entity can be forwarded from that network entity to another network entity. For example, interface entity 850 can receive a certificate bundle from a distribution service, as shown in the reference. Figure 8A and Figure 8B As described. Reference Figure 8C and Figure 8D The described interface entity 850 can be a reference. Figure 8A and Figure 8B The network entity described. In one example, interface entity 850 could be a smartNIC. Interface entity 850 could include a forwarding module 858. The forwarding module could forward certificate bundles to other network entities associated with the virtual cloud network. For example, as... Figure 8C and Figure 8D As shown, the certificate bundle can be forwarded by interface entity 850 (such as by forwarding module 858 of interface entity 850) to overlay entity 806. In one example, overlay entity 806 can be an overlay host.

[0177] Operation 800 may include the operations of a forwarding entity (such as interface entity 850) shown on forwarding entity timeline 854. Additionally, or alternatively, operation 800 may include the operations of a receiving entity (such as overlay entity 852) shown on receiving entity timeline 856. A forwarding entity (such as interface entity 850) may include a forwarding module 858. Forwarding module 858 may perform at least a portion of the operations described with reference to forwarding entity timeline 854. A receiving entity (such as overlay entity 852) may include an update module 860. Update module 860 may perform at least a portion of the operations described with respect to receiving entity timeline 856.

[0178] refer to Figure 8CAt time 862, overriding entity 852 may request a certificate bundle. For example, update module 860 of overriding entity 852 may send a request for the certificate bundle to forwarding module 858 of interface entity 850. At time 864, forwarding module 858 may receive the request for the certificate bundle. In response to the request, forwarding module 858 may send the certificate bundle to overriding entity 852, such as to update module 860 of overriding entity 852. At time 866, update module 860 may install the certificate bundle. Overriding entity 852 may use the CA certificate in the certificate bundle to authenticate network entities associated with the virtual cloud network.

[0179] The certificate bundle sent to the overriding entity 852 at time 864 could be the initial certificate bundle, such as the initial certificate bundle distributed to the interface entity at time 816, as shown in the reference. Figure 8A As described, forwarding module 858 can forward certificate bundles to various overriding entities. Overriding entities can use the CA certificate in the certificate bundle to verify network entities associated with the virtual cloud network. In one example, update module 860 of overriding entity 852 can periodically request forwarding module 858 to forward certificate bundles. Forwarding module 858 can respond to periodic requests by sending certificate bundles to update module 860, for example, when an updated certificate bundle is available.

[0180] At time 868, interface entity 850 can receive the intermediate certificate bundle. Receiving the intermediate certificate bundle at time 868 can be done in accordance with reference to [reference needed]. Figure 8A The description of receiving the intermediate certificate bundle from the distribution service at time 826 coincides with time 802. At time 870, for example, according to the reference... Figures 9A to 9C As described in one or more operations, interface entity 850 can verify the intermediate certificate bundle. In response to the verification of the intermediate certificate bundle, interface entity 850 can install the intermediate certificate bundle at time 872.

[0181] At time 874, forwarding module 858 may send the intermediate certificate bundle to overriding entity 852, such as to update module 860. In one example, forwarding module 858 may push the intermediate certificate bundle to overriding entity 852 in response to interface entity 850 installing the intermediate certificate bundle. Additionally, or alternatively, at time 876, update module 860 may send a request to forwarding module 858, and forwarding module 858 may send the intermediate certificate bundle to update module 860 in response to the request. Update module 860 may receive the intermediate certificate bundle in response to the request at time 876.

[0182] At time 878, update module 860 can verify the intermediate certificate bundle received from forwarding module 858. In one example, this can be done according to reference... Figures 9A to 9C One or more operations described verify the intermediate certificate bundle. In response to the verification of the intermediate certificate bundle, update module 860 may install the intermediate certificate bundle at time 880. After the intermediate certificate bundle is verified and installed by update module 860, interface entity 850 may begin to use at least some of the CA certificates in the intermediate certificate bundle to authenticate other network entities associated with the virtual cloud network.

[0183] Now for reference Figure 8D At time 882, interface entity 850 can receive the second certificate bundle. Receiving the second certificate bundle at time 882 can be done in conjunction with, as referenced... Figure 8B The described time 842 coincides with the receipt of the second certificate bundle from the distribution service 802. At time 884, for example, according to reference... Figures 9A to 9C As described in one or more operations, interface entity 850 can verify the second certificate bundle. In response to the verification of the second certificate bundle, interface entity 850 can install the second certificate bundle at time 886.

[0184] At time 892, forwarding module 858 may receive a request for an updated certificate bundle from update module 860 of overriding entity 852. The request from update module 860 may be one of a series of periodic requests. At time 890, forwarding module 858 may send a second certificate bundle to update module 860 of overriding entity 852. Update module 860 may receive the second certificate bundle at time 892 in response to the request from forwarding module 858. Additionally, or alternatively, forwarding module 858 may push the second certificate bundle to update module 860 in response to the second certificate bundle being installed by interface entity 850.

[0185] In response to receiving the second certificate bundle, at time 846 and at time 892, update module 860 can verify the second certificate bundle received from forwarding module 858. In one example, this can be done according to reference... Figures 9A to 9COne or more operations described verify the second certificate bundle. In response to the verification of the second certificate bundle, update module 860 may install the second certificate bundle at time 896. After the second certificate bundle is verified and installed by update module 860, interface entity 850 may begin to use at least some of the CA certificates in the second certificate bundle to authenticate other network entities associated with the virtual cloud network. For example, a new CA certificate may be used to authenticate network entities. Additionally, or alternatively, any CA certificates included in the second certificate bundle from a subset of the first CA certificates (e.g., those not replaced by a new CA certificate) may be used to authenticate network entities.

[0186] B. Distribution Operation Processing

[0187] Now for reference Figures 9A to 9C This further describes the operations involved in distributing certificate bundles to network entities associated with the virtual cloud network. (See reference) Figures 9A to 9C One or more operations 900 described can be modified, rearranged, or omitted entirely. Therefore, refer to Figures 9A to 9C The specific order of operations 900 described should not be construed as limiting the scope of one or more embodiments. In one example, reference... Figures 9A to 9C The described operation 900 can be compared with the reference. Figure 7 The described operation corresponds to phase 1 and / or phase 3. Additionally, or alternatively, refer to... Figures 9A to 9C The described operation 900 can be compared with the reference. Figures 8A to 8D The description corresponds to one or more operations.

[0188] refer to Figures 9A to 9C This describes the operations associated with network entities. (900) Figure 9A As shown, operation 900 may include, at block 902, receiving a first certificate bundle including a first set of updated CA certificates at a network entity associated with the virtual cloud network. The network entity may be a base entity, interface entity, or overlay entity. The first set of updated CA certificates may include an earlier CA certificate and a new CA certificate. In one example, the first certificate bundle may be transmitted to the network entity in response to an update request issued by the network entity.

[0189] At box 904, operation 900 may include applying verification processing to a first entity certificate issued to the network entity before the network entity receives the first certificate bundle. The first entity certificate may be received and installed before receiving the first certificate bundle. The verification processing may include verification by the network entity of a first certificate chain comprising the first entity certificate and a first set of updated CA certificates contained in the first set of CA certificates. The first CA certificate may be an earlier CA certificate. The first certificate chain may be configured to include one or more verification operations (including, for example, references...). Figure 9C The verification process described herein is used for verification. At block 906, operation 900 may include determining whether the verification operation at block 904 resulted in the determination that the first certificate chain is valid. If the first certificate chain is determined to be valid, operation 900 may proceed to block 908. If the first certificate chain is determined to be invalid, operation 900 may proceed to block 910.

[0190] At block 908, in response to determining that the first certificate chain is valid, operation 900 may include installing the first certificate bundle on the storage medium associated with the network entity. After installing the first certificate bundle, operation 900 may include, at block 912, the network entity authenticating at least one additional network entity associated with the virtual cloud network using the first certificate bundle. Additionally, or alternatively, at block 910, operation 900 may include rejecting the first certificate bundle in response to determining that the first certificate chain is invalid. In response to rejecting the first certificate bundle, operation 900 may include, at block 914, the network entity authenticating at least one additional network entity associated with the virtual cloud network using an earlier certificate bundle.

[0191] In one example, operation 900 may include forwarding a certificate bundle to another network entity, such as an overlay entity. Operation 900 may include receiving a first certificate bundle request from the overlay entity at the network entity. The first certificate bundle request may be received after verifying a first certificate chain. The network entity may be an interface entity. In response to the first certificate bundle request, operation 900 may include transferring the first certificate bundle, for example, from the interface entity to the overlay entity. Operation 900 may also include the overlay entity verifying a first overlay entity certificate chain, which includes a first entity certificate issued to the overlay entity and a CA certificate (such as an earlier CA certificate) from a first set of updated CA certificates. In one example, the first entity certificate issued to the overlay entity may be a first instance primary certificate. In response to the overlay entity verifying the first overlay entity certificate chain, operation 900 may include installing the first certificate bundle. Operation 900 may also include the overlay entity using the first certificate bundle to authenticate at least one additional entity associated with the virtual cloud network.

[0192] In one example, operation 900 may include transmitting a second entity certificate (such as a second instance master certificate) issued to the overriding entity. The second instance master certificate may be transmitted from the interface entity to the overriding entity, for example, after the interface entity has verified the second certificate chain. Operation 900 may include the overriding entity installing the second instance master certificate.

[0193] Now for reference Figure 9B This further describes the operations associated with network entities 900. In one example, refer to... Figure 9B The described operation can be obtained from Figure 9A Continue with box 912. (For example...) Figure 9B As shown, operation 900 may include, at block 916, receiving a second entity certificate issued to the network entity and installing the second entity certificate in a storage medium associated with the network entity. The second entity certificate may have already been issued by a CA and corresponds to a new CA certificate in a first certificate bundle distributed to and installed by the network entity, for example, as referenced... Figure 9A As described. At box 918, operation 900 may include receiving a second certificate bundle at a network entity, the second certificate bundle including a second set of updated CA certificates reflecting an update to the first set of updated CA certificates, including the removal of the first CA certificate. The first CA certificate may be an earlier CA certificate. The second set of updated CA certificates may include a new CA certificate. Additionally, or alternatively, the second set of updated CA certificates may include earlier CA certificates in the second certificate bundle that have not yet been replaced by the new CA certificates. In one example, the second certificate bundle may be transmitted to the network entity in response to an update request issued by the network entity.

[0194] At box 920, operation 900 may include applying verification processing to a second entity certificate based on a second certificate bundle. This verification processing may include verification by the network entity of a second certificate chain comprising the second entity certificate and a second CA certificate from a second set of updated CA certificates. The second CA certificate may be a new CA certificate or an earlier CA certificate in the second certificate bundle that has not yet been replaced by the new CA certificate. The second certificate chain may be configured to include one or more verification operations (including, for example, references...). Figure 9C The verification process described is used to perform verification.

[0195] At box 922, operation 900 may include determining whether the verification operation at box 920 resulted in the determination that the second certificate chain is valid. If the second certificate chain is valid, operation 900 may proceed to box 924. If the second certificate chain is invalid, operation 900 may proceed to box 926.

[0196] At block 924, in response to determining that the second certificate chain is valid, operation 900 may include installing the second certificate bundle into the storage medium associated with the network entity. After installing the second certificate bundle, operation 900 may include, at block 928, the network entity authenticating at least one additional network entity associated with the virtual cloud network using the second certificate bundle. Additionally, or alternatively, at block 926, operation 900 may include rejecting the second certificate bundle in response to determining that the second certificate chain is invalid. In response to rejecting the second certificate bundle, operation 900 may include, at block 930, the network entity authenticating at least one additional network entity associated with the virtual cloud network using the earlier certificate bundle. In one example, after verifying the second certificate chain, operation 900 may include unloading the first entity certificate.

[0197] In one example, operation 900 may include receiving a request for a second certificate bundle from an overlay entity at the interface entity. In response to the second certificate bundle request, operation 900 may include transferring the second certificate bundle, for example, from the interface entity to the overlay entity. Operation 900 may also include the overlay entity verifying a second overlay entity certificate chain, which includes a second entity certificate (such as a second instance master certificate) issued to the overlay entity and a CA certificate from a second set of updated CA certificates. In response to the overlay entity verifying the second overlay entity certificate chain, operation 900 may include installing the second certificate bundle. Operation 900 may also include unloading a first entity certificate, for example, after verifying the second overlay entity certificate chain. Operation 900 may also include the overlay entity using the second certificate bundle to authenticate at least one additional entity associated with the virtual cloud network. In one example, the interface entity may use the second certificate bundle to authenticate the overlay entity, and / or the overlay entity may use the second certificate bundle to authenticate the interface entity.

[0198] Now for reference Figure 9C The document further describes example operation 900 associated with a network entity. Operation 900 may include operations associated with verifying a certificate chain, such as referencing... Figure 9A The description refers to verifying the first certificate chain and / or referencing at box 904. Figure 9B The description refers to verifying the second certificate chain at box 920. Figure 9C The operations shown can be repeated for each signature-key pair in the certificate chain. The certificate chain can include a set of signature-key pairs, and each signature-key pair in the set can include a public key corresponding to the issuer's certificate and a digital signature corresponding to the recipient's certificate.

[0199] In one example, the certificate chain may include a digital signature in the entity certificate and a signature-key pair of the public key corresponding to the CA certificate of the CA that issued the entity certificate. In one example, the CA that issued the entity certificate may be a root CA, and the CA certificate may be a root CA certificate. In one example, the CA that issued the entity certificate may be an intermediate CA, and the CA certificate may be an intermediate CA certificate. Additionally, or alternatively, the certificate chain may include a digital signature in the intermediate CA certificate and a signature-key pair of the public key corresponding to the CA certificate of the CA that issued the intermediate CA certificate. In one example, the intermediate CA may be the issuer of the entity certificate, and the root CA may be the issuer of the intermediate CA certificate.

[0200] like Figure 9C The diagram illustrates operation 900 for a corresponding signature-key pair in a certificate chain. As shown, operation 900 may include, at box 932, at least using the public key corresponding to the issuer certificate to compute a decryption hash value for decrypting the digital signature corresponding to the recipient certificate. For one signature-key pair, the issuer certificate may be a CA certificate, and the recipient certificate may be an entity certificate. In one example, the CA certificate may be an intermediate CA certificate. For another signature-key pair, the recipient certificate may be a CA certificate, and the issuer certificate may be a higher-level CA certificate. For example, the recipient certificate may be an intermediate CA certificate, and the issuer certificate may be a root CA certificate.

[0201] At box 934, operation 900 may include calculating a signature hash value by applying a hash function to the digital signature, and at box 936, operation 900 may include comparing the decrypted hash value with the signature hash value. At box 938, operation may include determining whether the decrypted hash value matches the signature hash value. For a decrypted hash value that does not match the signature hash value, operation may include, at box 940, determining that the certificate chain is invalid. In one example, when the decrypted hash value does not match the signature hash value used for any signature-key pair in the certificate chain, the certificate chain may be determined to be invalid at box 940. For a decrypted hash value that matches the signature hash value, operation 900 may include, at box 942, determining that the signature-key pair is valid. At box 944, operation 900 may include, at box 944, determining whether the certificate chain includes another signature-key pair. For a signature chain that includes another signature-key pair, operation 900 may return to box 932. For a signature chain that does not include another signature-key pair, operation 900 is possible, and at box 946, the certificate chain is determined to be valid. The certificate chain is valid when the decrypted hash value matches the signature hash value of each signature-key pair.

[0202] 10. Other matters; extension

[0203] The embodiments are directed to a system having one or more devices, which include a hardware processor and are configured to perform any of the operations described herein and / or any of the following claims.

[0204] The embodiments relate to a system that includes components that perform any of the operations described herein and / or set forth in any of the following claims.

[0205] In an embodiment, the non-transitory computer-readable storage medium includes instructions that, when executed by one or more hardware processors, cause to perform any of the operations described herein and / or any of the claims.

[0206] Further aspects of the currently disclosed subject matter are provided by the following terms:

[0207] A non-transitory computer-readable medium including instructions that, when executed by one or more hardware processors, cause the following operations to be performed: distributing a first certificate bundle to a plurality of entities associated with a virtual cloud network, the first certificate bundle including a first set of updated Certificate Authority (CA) certificates, the first set of updated CA certificates including a first subset of earlier CA certificates and a second subset of new CA certificates; determining a verification indicator indicating that at least a threshold portion of the plurality of entities has (i) verified a certificate chain including at least one CA certificate from the first set of updated CA certificates, or (ii) installed the first certificate bundle in response to verification of the certificate chain. The package, wherein for a first entity among the plurality of entities, the certificate chain includes a first entity certificate and a first CA certificate issued to the first entity, wherein the first CA certificate is an earlier CA certificate in a first subset of earlier CA certificates; in response to determining a verification indicator: activating one or more new CA certificates in a second subset of new CA certificates; generating a second certificate bundle reflecting an update to the first certificate bundle, wherein the update includes removing the first subset of earlier CA certificates, and wherein the second certificate bundle includes a second set of updated CA certificates containing the second subset of new CA certificates; and distributing the second certificate bundle including the second set of updated CA certificates to the plurality of entities.

[0208] The medium of any provision herein, wherein the verification indicator includes at least one of the following: a time period has elapsed; a notification from or representing a threshold portion of the plurality of entities; or an update request from or representing a threshold portion of the plurality of entities.

[0209] The medium of any provision herein, wherein the operation further includes: receiving a first update request from a first entity; wherein distributing a first certificate bundle to the plurality of entities includes transmitting the first certificate bundle to the first entity in response to the first update request.

[0210] The medium of any provision herein, wherein the operation further includes: receiving a second update request from a first entity; wherein distributing a second certificate bundle to the plurality of entities includes transmitting the second certificate bundle to the first entity in response to the second update request.

[0211] The operation further includes, in any provision of this document, distributing a second entity certificate issued to the first entity to the first entity after distributing the first certificate bundle; and, prior to distributing the second certificate bundle, determining an installation indicator that indicates the first entity has installed the second entity certificate in the entity storage medium associated with the first entity.

[0212] The medium of any of the provisions herein, wherein the installation instruction includes at least one of the following: after a period of time; notification from the first entity; or a request for update.

[0213] A non-transitory computer-readable medium including instructions that, when executed by one or more hardware processors, cause to perform the following operations: receiving a first certificate bundle at a first entity associated with a virtual cloud network, the first certificate bundle including a first set of updated Certificate Authority (CA) certificates and a first set of current CA certificates; installing the first certificate bundle in a first storage medium associated with the first entity, and having the first entity use the first certificate bundle to authenticate at least a second entity associated with the virtual cloud network; receiving a second entity certificate issued to the first entity at the first entity, and installing the second entity certificate in the first storage medium associated with the first entity; receiving a second certificate bundle at the first entity including the first set of updated CA certificates but not including the first set of current CA certificates; installing the second certificate bundle in the first storage medium associated with the first entity, and using the second certificate bundle to authenticate at least a third entity associated with the virtual cloud network.

[0214] According to one or more embodiments, any combination of the features and functions described herein may be used. In the foregoing description, embodiments have been described with reference to numerous specific details that may vary depending on the implementation. Accordingly, this specification and the drawings should be viewed in an illustrative rather than restrictive sense. The unique and exclusive reference to the scope of the invention, and what the applicant intends to define as the scope of the invention, is the literal and equivalent scope of the set of claims issued herein, and should be in the specific form at the time of issuance of those claims, including any subsequent corrections.

[0215] All references cited in this article, including publications, patent applications and patents, are incorporated into this article by reference to the same extent as if each reference individually and specifically indicated to be incorporated by reference and elaborated in full in this article.

Claims

1. A non-transitory computer-readable medium comprising instructions that, when executed by one or more hardware processors, cause to perform operations including: Receive a first certificate bundle at a first entity associated with the virtual cloud network, the first certificate bundle including a first set of updated Certificate Authority (CA) certificates; The verification process is applied to the first entity certificate based on the first certificate bundle. The first entity certificate is issued to the first entity before the first entity receives the first certificate bundle. The verification process includes: The first entity verifies the first certificate chain, which includes the first entity certificate and the first CA certificate included in the first set of updated CA certificates; In response to verifying the first certificate chain, a first certificate bundle is installed in a first storage medium associated with the first entity, and the first entity uses the first certificate bundle to authenticate at least one additional entity associated with the virtual cloud network.

2. The medium as described in claim 1, Verifying the first certificate chain includes verifying a set of signature-key pairs; Each of the signature-key pairs in the set includes a public key corresponding to the issuer's certificate and a digital signature corresponding to the recipient's certificate. Verifying the set of signature-key pairs includes: For the corresponding signature-key pair in the set of signature-key pairs: (a) The decryption hash value must be calculated at least by decrypting the digital signature using the public key. (b) The signature hash value is calculated at least by applying a hash function to the digital signature. (c) Compare the decrypted hash value with the signed hash value, and (d) Determine if there is a match between the decrypted hash value and the signature hash value, and For each signature-key pair in the set of signature-key pairs, repeat elements (a), (b), (c), and (d); Wherein, for at least one of the signature-key pairs in the set, the recipient certificate is the first entity certificate, and wherein, for at least one of the signature-key pairs in the set, the issuer certificate is the first CA certificate in the set of CA certificates.

3. The medium as described in claim 2, The signature-key pair set includes a first signature-key pair and a second signature-key pair; For the first signature-key pair, the recipient certificate is the first entity certificate and the issuer certificate is the first intermediate CA certificate; and For the second signature-key pair, the recipient certificate is the first intermediate CA certificate, and the issuer certificate is one of the following: the first root CA certificate or the second intermediate CA certificate.

4. The medium as described in claim 3, The signature-key pair set includes a first signature-key pair, a second signature-key pair, and a third signature-key pair; For the second signature-key pair, the issuer certificate is the second intermediate CA certificate; and For the third signature-key pair, the recipient certificate is the second intermediate CA certificate, and the issuer certificate is the first CA certificate.

5. The medium of claim 1, wherein the operation further comprises: After verifying the first certificate chain, the second entity certificate issued to the first entity is received at the first entity, and the second entity certificate is installed in the first storage medium associated with the first entity. A second certificate bundle is received at a first entity, comprising a second set of updated CA certificates reflecting an update to the first set of updated CA certificates, wherein the update includes the removal of the first CA certificate. The second certificate chain is verified by the first entity, wherein the second certificate chain includes the second entity certificate and the second CA certificate in the second set of updated CA certificates; In response to verifying the second certificate chain, the second certificate bundle is installed in the first storage medium associated with the first entity, and the second certificate bundle is used to authenticate at least one additional entity associated with the virtual cloud network.

6. The medium as described in claim 5, Verifying the second certificate chain includes verifying a set of signature-key pairs; Each of the signature-key pairs in the set includes a public key corresponding to the issuer's certificate and a digital signature corresponding to the recipient's certificate. Verifying the set of signature-key pairs includes: For the corresponding signature-key pair in the set of signature-key pairs: (a) The decryption hash value must be calculated at least by decrypting the digital signature using the public key. (b) The signature hash value is calculated at least by applying a hash function to the digital signature. (c) Compare the decrypted hash value with the signed hash value, and (d) Determine if there is a match between the decrypted hash value and the signature hash value, and For each signature-key pair in the set of signature-key pairs, repeat elements (a), (b), (c), and (d); Wherein, for at least one of the signature-key pairs in the set, the recipient certificate is a first entity certificate, and wherein, for at least one of the signature-key pairs in the set, the issuer certificate is a second CA certificate in the set of CA certificates.

7. The medium of claim 6, wherein the operation further comprises: Transmit the first update request to the certificate bundle distribution service; In response to the first update request, a first certificate bundle including the first set of updated CA certificates is received; After verifying the first certificate chain, a second update request is transmitted to the certificate bundle distribution service; as well as In response to a second update request, a second certificate bundle is received, which includes a second set of updated CA certificates.

8. The medium of claim 6, wherein the operation further comprises: After verifying the second certificate chain, the first entity certificate is unloaded from the first storage medium associated with the first entity.

9. The medium of claim 1, wherein the operation further comprises: Before receiving the first certificate bundle, the first entity certificate is received at the first entity and installed in the first storage medium associated with the first entity.

10. The medium as described in claim 1, The first certificate chain includes one or more signature-key pairs, wherein each of the one or more signature-key pairs includes a public key and a digital signature, and Verifying the first certificate chain includes, for each of the one or more signature-key pairs: The decryption hash value is calculated at least by decrypting the digital signature of the signature-key pair using the public key of the signature-key pair; The signature hash value is calculated at least by applying a hash function to the digital signature of the signature-key pair; Compare the decrypted hash value with the signature hash value; Determine if there is a match between the decrypted hash value and the signature hash value; as well as The first certificate chain is designated as valid at least in part based on the fact that the decryption hash value for each of the one or more signature-key pairs has been determined to match the signature hash value.

11. The medium of claim 10, wherein the one or more signature-key pairs comprise: An intermediate signature-key pair, comprising an intermediate public key and an intermediate digital signature of an intermediate CA, wherein the intermediate public key corresponds to an intermediate CA certificate, and the first entity certificate includes the intermediate digital signature of the intermediate CA; and The root signature-key pair includes the root public key and root digital signature of the root CA, the root public key corresponds to the first CA certificate, and the intermediate CA certificate includes the root digital signature of the root CA.

12. The medium of claim 10, wherein the one or more signature-key pairs comprise: The root signature-key pair includes the root public key and root digital signature of the root CA, the root public key corresponds to the first CA certificate, and the first entity certificate includes the root digital signature of the root CA.

13. The medium of claim 1, wherein the operation further comprises: Receive a second certificate bundle, including a second set of updated CA certificates, at a second entity associated with the virtual cloud network; The second certificate chain is determined to be invalid, wherein the second certificate chain includes a second entity certificate that has been issued to the second entity and at least one second CA certificate from the second set of updated CA certificates; In response to determining that the second certificate chain is invalid, the second certificate bundle is rejected, and the earlier certificate bundle is used to authenticate at least one additional entity associated with the virtual cloud network.

14. The medium of claim 1, wherein the operation further comprises: After verifying the first certificate chain, a first certificate bundle request is received at the first entity from the overlay entity associated with the virtual cloud network, the virtual cloud network including the base network and the overlay network, wherein the overlay entity resides on the overlay network, and wherein the first entity includes an interface entity that provides a communication interface between the base network and the overlay entity. In response to the first certificate bundle request, the interface entity transmits the first certificate bundle, which includes the first set of updated CA certificates, from the overriding entity. The second certificate chain is verified by the overriding entity, wherein the second certificate chain includes the first instance master certificate that has been issued to the overriding entity and the second CA certificate in the first set of updated CA certificates; In response to verifying the second certificate chain, the first certificate bundle is installed in a second storage medium associated with the overlay entity, and the overlay entity uses the first certificate bundle to authenticate at least one additional entity associated with the virtual cloud network.

15. The medium of claim 14, wherein the operation further comprises: After verifying the second certificate chain, the second instance master certificate issued to the overlay entity is received at the overlay entity, and the second instance master certificate is installed in the second storage medium associated with the overlay entity. Receive a second certificate bundle request from the overriding entity at the first entity; In response to a second certificate bundle request, a second certificate bundle including a second set of updated CA certificates is transmitted from the interface entity to the overlay entity. The second set of updated CA certificates reflects an update to the first set of updated CA certificates, wherein the update includes the removal of the first certificate. Verify the third certificate chain, which includes the second instance master certificate and the third CA certificate in the second set of updated CA certificates; In response to verifying the third certificate chain, a second certificate bundle is installed in a second storage medium associated with the overlay entity, and the second certificate bundle is used to authenticate at least one additional entity associated with the virtual cloud network.

16. The medium of claim 15, wherein the operation further comprises: After verifying the third certificate chain, the first instance master certificate is unloaded from the second storage medium associated with the overriding entity.

17. The medium of claim 15, wherein the operation further comprises at least one of the following: The first entity uses a second certificate bundle to authenticate the overriding entity; or The overriding entity uses a second certificate bundle to authenticate the first entity.

18. The medium of claim 1, wherein the operation further comprises: Receive the first certificate bundle, which includes the first set of updated CA certificates, at the underlying entity associated with the virtual cloud network; The second certificate chain is verified by the primary entity, wherein the second certificate chain includes the first service master certificate that has been issued to the primary entity and the second CA certificate in the first set of updated CA certificates; In response to verifying the first primary certificate chain, a first certificate bundle is installed in a second storage medium associated with the primary entity, and the primary entity uses the first certificate bundle to authenticate at least one additional entity associated with the virtual cloud network.

19. The medium of claim 18, wherein the operation further comprises: After verifying the second certificate chain, the second service master certificate issued to the base entity is received at the base entity and installed in the second storage medium associated with the base entity; At the base entity, a second certificate bundle is received, which includes a second set of updated CA certificates reflecting an update to the first set of updated CA certificates, wherein the update includes the removal of the second CA certificate. Verify the third certificate chain, which includes the second service master certificate and the third CA certificate in the second set of updated CA certificates; In response to verifying the second primary certificate chain, a second certificate bundle is installed in a second storage medium associated with the primary entity, and the second certificate bundle is used to authenticate at least one additional entity associated with the virtual cloud network.

20. The medium of claim 19, wherein the operation further comprises: After verifying the third certificate chain, the first service master certificate is unloaded from the second storage medium associated with the underlying entity.

21. The medium of claim 19, wherein the operation further comprises at least one of the following: The first entity uses a second certificate bundle to authenticate the underlying entity; or The primary entity uses the second certificate bundle to authenticate the primary entity.

22. The medium of claim 1, wherein the first set of updated CA certificates further includes a second CA certificate, wherein the first CA certificate is the previous CA certificate and the second CA certificate is the subsequent CA certificate.

23. The medium of claim 22, wherein the first CA certificate is issued by a first CA certificate authority, and the second CA certificate is issued by a second CA certificate authority.

24. The medium of claim 1, wherein the first entity comprises one of the following: a base layer entity, an overlay entity, or an interface entity.

25. A method comprising the operation as described in any one of claims 1-24.

26. A system comprising: At least one hardware processor; The system is configured to perform the operation as described in any one of claims 1-24 using the at least one hardware processor.

27. A computer program product comprising instructions that, when executed by one or more hardware processors, cause to perform the operations described in any one of claims 1-24.

Images