Techniques to establish trust in cluster of edge devices

Abstract

Techniques for establishing trust in a cluster of edge devices are disclosed. An edge device cloud service may associate a first cloud computing edge device with a cluster of cloud computing edge devices and provision the first cloud computing edge device with a master encryption key. The edge device cloud service may associate a second cloud computing edge device with the fleet and provide the second cloud computing edge device with the master encryption key and the first common encryption key. The first cloud computing edge device may receive encrypted message data including a second public encryption key from the second cloud computing edge device. The first cloud computing edge device may decrypt the encrypted message data using a master encryption key stored in the first key store and update the first key store with a second common encryption key.

Description

[0001] Cross-references to related applications

[0002] This international application claims priority and benefit to U.S. Patent Application 18 / 373,226, filed on September 26, 2023, entitled “TECHNIQUES FORESTABLISHING TRUST IN A CLUSTER OF EDGE DEVICES”, the entire contents of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0003] This disclosure generally relates to clusters of cloud computing edge devices. More specifically, this disclosure relates to establishing trust between a cluster of cloud computing edge devices and additional edge devices subsequently added to that cluster. Background Technology

[0004] In cloud computing, processing and storage are typically performed by one or more service providers located in a centralized location. Data can be received from customers at the centralized location, processed there, and then the processed (or other) data can be transmitted back to the customers. However, having a centralized location for cloud infrastructure components may not be ideal in various scenarios. For example, a conventional centralized system is not ideal when there are hundreds or thousands of Internet of Things (IoT) devices transmitting data to a central server, especially when these IoT devices are not geographically close to the cloud infrastructure computing devices. These IoT devices can be considered to be at the "edge" because they are not close to the central server.

[0005] Furthermore, there may be other instances where centralized locations for cloud components are less than ideal. For example, if data is collected in disconnected areas or locations without internet connectivity (e.g., remote locations) (e.g., collected by IoT devices). Current centralized cloud computing environments may not meet the time-sensitive requirements of streaming data due to the inherent latency of their WAN connections. Remotely generated data may require processing much faster than allowed by conventional centralized cloud computing systems (e.g., to detect anomalies). Therefore, managing traditional cloud computing environments that rely on centralized components presents challenges. Summary of the Invention

[0006] Embodiments of this disclosure relate to establishing trust among several devices providing cloud computing or other distributed computing services at an “edge” location. Specifically, a distributed computing system may include a “fleet” of edge devices that can be deployed as part of a cluster at an edge location and perform operations to provide cloud computing services without connecting to a cloud service provider (CSP) or other cloud service for post-deployment device configuration and management. As used herein, the term “fleet” may refer to a logical abstraction of an aggregation of edge devices as part of the same cluster, while the terms “edge device,” “edge computing device,” and “cloud computing edge device” may refer to the various edge devices described herein. A CSP may be responsible for configuring the edge devices prior to delivery to a customer location, but edge devices may be deployed to customer locations without public network connectivity or with restricted network connectivity that prevents communication with the CSP after deployment. Because additional edge devices can be provisioned and deployed to a fleet, existing edge devices may require a way to determine whether additional edge devices can be trusted when establishing secure communication channels between them within the cluster.

[0007] One embodiment relates to a method performed by a distributed computing system comprising multiple cloud computing edge devices that can be configured by an edge device cloud service and subsequently operate in a distributed computing cluster without communicating with the edge device cloud service. As used herein, the edge device cloud service may include one or more programs, processes, or applications executed by one or more processors of a computer system to perform operations of the edge device cloud service. The method may include the edge device cloud service associating a first cloud computing edge device with a cluster of cloud computing edge devices. Associating the first cloud computing edge device with the cluster may include obtaining a first public encryption key for the first cloud computing edge device. The cluster may be characterized by a master encryption key and a public encryption key obtained from the cloud computing edge devices in the cluster. As used herein, the master encryption key may be characterized by associating the cluster with cluster data maintained by a cloud service provider. For example, the cloud service provider may maintain a master encryption key associated with each cluster of cloud computing edge devices, where this association is provided by a data table. The edge device cloud service may maintain the master encryption key and the public encryption key in a data repository. The method may also include the edge device cloud service provisioning the first cloud computing edge device with the master encryption key. The first cloud computing edge device may store the master encryption key in a first key repository. The method may further include an edge device cloud service associating a second cloud computing edge device with the cluster. The edge device cloud service can obtain a second public encryption key from the second cloud computing edge device and then supply it to the second cloud computing edge device, which carries a master encryption key and a first public encryption key. The second cloud computing edge device can store the master encryption key and the first public encryption key in a second key repository. The method also includes the first cloud computing edge device receiving encrypted message data from the second cloud computing edge device, the encrypted message data including the second public encryption key. The encrypted message data can be generated by the second cloud computing edge device using the master encryption key. The first cloud computing edge device can decrypt the encrypted message data using the master encryption key stored in the first key repository and update the first key repository with the second public encryption key.

[0008] Another embodiment relates to a distributed computing system configured with one or more processors and one or more memories storing computer-executable instructions, which, when executed by the one or more processors, cause the computing cluster to perform the methods described in the preceding paragraphs.

[0009] Another embodiment relates to a non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors of a computing cluster, cause the computing cluster to perform the methods disclosed herein. Furthermore, embodiments can be implemented using a computer program product comprising computer programs / instructions that, when executed by a processor, cause the processor to perform any of the methods described in this disclosure. Attached Figure Description

[0010] Figure 1 This is a block diagram of an example high-level architecture of a cloud infrastructure edge computing device according to at least one embodiment, based on some embodiments.

[0011] Figure 2 This is a block diagram of an example architecture for connecting a user computing device to a cloud infrastructure edge computing device, according to at least one embodiment of some embodiments.

[0012] Figure 3 This is a block diagram of an example computer architecture for a cloud infrastructure edge computing device according to at least one embodiment, based on some embodiments.

[0013] Figure 4 It is a block diagram depicting a distributed computing cluster including one or more edge computing devices according to at least one embodiment, based on some embodiments.

[0014] Figure 5 This is a block diagram depicting a distributed computing system comprising one or more edge devices, according to some embodiments, which are managed by a CSP as part of a cluster.

[0015] Figure 6 This is a block diagram depicting a fleet of edge devices supplied by a CSP according to some embodiments.

[0016] Figure 7 This is a block diagram depicting the configuration of an edge device with an encryption key as part of associating an edge device with a cluster, according to some embodiments.

[0017] Figure 8 It is a block diagram depicting communication between two edge devices in a distributed computing cluster according to some embodiments.

[0018] Figure 9 This is an example process for the provisioning and deployment of two edge devices according to some embodiments.

[0019] Figure 10 This is an example process for establishing trust between two edge devices in a distributed computing system, according to some embodiments.

[0020] Figure 11 This is a block diagram illustrating a pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0021] Figure 12 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0022] Figure 13 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0023] Figure 14 This is a block diagram illustrating another pattern for implementing a cloud infrastructure-as-a-service system according to at least one embodiment.

[0024] Figure 15 This is a block diagram illustrating an example computer system according to at least one embodiment. Detailed Implementation

[0025] Introduction

[0026] In some examples, cloud-integrated edge services (e.g., implemented in edge computing devices) can be indispensable for addressing the expectation of running time-sensitive cloud infrastructure applications outside of centralized data centers (e.g., data centers of cloud infrastructure service providers). Such edge computing devices can deliver compute and storage at the edge and / or in disconnected locations (e.g., remote locations isolated from centralized data centers and lacking public / private network connectivity (e.g., internet connectivity, VPN connectivity, dedicated connections, etc.)) to enable low-latency processing at or near the data generation and ingestion points. In some instances, a fleet of portable (which may be hardened for protection) server nodes (e.g., a fleet of edge devices) can be configured to physically bring cloud infrastructure services to remote locations where cloud technologies were previously considered technically infeasible or prohibitively expensive to implement.

[0027] For customers (e.g., users), edge computing devices can act as an extension of their cloud infrastructure: virtual machines (VMs), containers, functions and data files, block volumes, or object storage services can also be delivered from the cloud infrastructure (e.g., the lease of a centralized cloud computing environment) with minimal or no modification, and the customer experience can remain unchanged from the centralized cloud computing experience. Furthermore, edge computing devices can be configured to implement both the control plane and the data plane as part of a cloud infrastructure service provider. The data plane can be configured to manage data storage, migration, processing, etc., while the control plane can be configured to control the various services and architectural components of the computing device. Once the edge computing device is properly connected to the customer's computing device (e.g., via a local area network (LAN)), the customer can leverage IaaS services (or at least a subset thereof) using the same SDKs and APIs as centralized cloud services.

[0028] Edge computing devices can be delivered to customers in a pre-configured form, so that the only actions customers may need to perform are connecting nodes to a network (e.g., a local / on-premises network accessible to the user's computing device), powering them on, and / or logging in. Devices can be pre-configured in various ways based on customer preferences / requests, or they can be one of various configurations (e.g., storage-centric, compute-centric, etc.). Nodes or clusters of nodes can be portable and designed to be mobile—when moved and reconfigured (or used in motion), the deployment continues to operate from where it was shut down (or continues to operate). Edge computing devices can also monitor wide area network (WAN) connectivity availability (e.g., the internet, etc.) and, once connected to the WAN, can synchronize customer and management data with the cloud.

[0029] Some potential use cases for edge computing devices include: storage and processing, compute and I / O intensive applications, machine learning, remote computing, low-latency databases and analytics, and data collection and migration. More specifically, edge devices can be used to store and process large volumes of image, video, audio, and IoT sensor data generated in environments where WAN connectivity is sometimes delayed or unavailable (e.g., in remote areas, offshore oil platforms, etc.). Once this data has been preprocessed, filtered, compressed, and / or protected, it can be transported or transmitted to a cloud service provider, where it can be further processed by centralized servers (e.g., traditional cloud service providers). The device can also be used for compute and I / O intensive applications where low latency is critical, such as tactical reconnaissance or 5G communications. It can also be used for machine learning, where models are trained in the cloud and run in disconnected locations to improve efficiency, intelligence, and / or productivity in manufacturing, document management, transportation, oil and gas extraction, and / or telecommunications. It can also be used for remote computing requiring enhanced security and robust data isolation. Furthermore, the device can be used for low-latency database and analytics workloads, with more applications being optimized over time. Additionally, the device can be used to collect and migrate large collections of object and database management system (DBMS) data to cloud service providers (e.g., at faster speeds and lower costs than WAN transmission).

[0030] Edge devices can natively support distributed cloud paradigms, where complex, multi-stage computing workflows can be decomposed into individual components that can then be deployed across the edge device's infrastructure, on-premises deployments, and / or the cloud. An example of this distributed workflow is illustrated in the following scenario: Massive amounts of data can be collected by edge computing nodes (e.g., disconnected edge computing devices) deployed on aircraft (e.g., military jets) in reconnaissance operations without internet access. This data is preprocessed in near real-time by machine learning models previously trained by the cloud service provider supplying the edge devices. Even the first pass of data processing by the model can detect significant anomalies and immediately alert personnel—for example, a bridge may be destroyed, requiring troops to reroute. When the aircraft lands, the edge computing device can physically connect to the network (e.g., an edge station possibly deployed at a makeshift airfield). The preprocessed, filtered, smaller dataset can be loaded into a cluster of edge computing device nodes at the edge station for final processing. The original edge computing device can be released and loaded onto another (or the same) aircraft, for example, to support the next mission. Once processing at the edge station is complete, a 3D map update can be published for immediate use. The changeset can then be uploaded from the edge station cluster to the data center and used to build future models, providing intelligent tactical predictions for operations such as reconnaissance.

[0031] It should be recognized that the following technologies can be used in a variety of contexts, such as telecommunications, oil and gas, healthcare, hospitality, agriculture, transportation, and logistics.

[0032] The embodiments described herein address these and other problems individually and collectively. Specifically, embodiments of this disclosure provide edge computing devices for cloud infrastructure.

[0033] Edge device architecture

[0034] Edge computing devices (sometimes referred to as "cloud edge devices" or "edge devices" for brevity) extend a user's centralized cloud rental by physically placing the customer's infrastructure and platform services where data is generated—at the edge, on-premises, or completely disconnected. Each deployment is created to address a specific customer need by supplying VM instance images and data from the customer's centralized cloud rental. These workloads remain fully functional even when offline because edge devices adapt to connectivity, operate in harsh environmental conditions, and are ready to sync with the cloud whenever connectivity is re-established.

[0035] Figure 1 This is a block diagram of an example high-level architecture of a cloud infrastructure edge computing device (e.g., edge device 100) according to at least one embodiment. An overview of the software and hardware components of edge device 100 is provided below.

[0036] In some examples, edge device 100 may include a containerization engine 102 (e.g., Docker, Kubernetes, etc.) configured to implement one or more containers (e.g., corresponding to one or more services 104A, 104B, 104C to 104N (collectively, "one or more services 104")). The containerization engine (e.g., containerization engine 102) may be a container orchestration system for automating the deployment, scaling, and management of computer applications. In some embodiments, the containerization engine may be configured to provide OS-level virtualization to deliver software in packages called containers. These containers may be isolated from each other and utilize their respective software, libraries, and configuration files, and may communicate with each other through well-defined channels. In some embodiments, services 104 may include any suitable number of services (e.g., one or more). These services may implement at least some portion of centralized cloud capabilities. Each service may operate independently or as a distributed cluster. Edge device 100 may also include hypervisor 106, which is configured to implement one or more virtual machines (e.g., virtual machines 108A, 108B, 108C to 108N, collectively referred to as "(one or more) virtual machines 108" or "VM 108").

[0037] In some examples, edge device 100 includes storage device 110 (e.g., object and / or block storage for storing local data). Edge device 100 includes operating system (OS) 112. In some embodiments, OS 112 may be optimized for and / or specific to execution on the edge device. OS 112 may be configured to manage the hardware of edge device 100 and support the data plane of services running on edge device 100. OS 112 may be configured to support specific deployment types (e.g., single edge device deployment, or specific edge device cluster configuration). OS 112 may be configured to protect the edge device by prohibiting direct access by clients.

[0038] In some embodiments, the edge device 100 may include hardware such as any suitable number of central processing units (CPUs) and / or storage drives. For example, Figure 1 The edge device 100 depicted may have one, two, or more CPUs, each processing unit having a different number of cores, and may include any number of storage drives (e.g., a 6.4 terabyte (TB) drive, etc.). As a non-limiting example, the edge device 100 may include block and / or object storage devices of any suitable size. The edge device 100 may include any suitable number of central processing units (CPUs), graphics processing units (GPUs), random access memory (RAM) of any suitable size, one or more ports (e.g., QSFP28, RJ45, dual-port, etc.), tamper-evident seals, or any suitable combination of the foregoing components.

[0039] In some examples, basic system functionality / services can be accessed via a custom-loaded RESTful API with Linux-based software. One or more virtual machines 108 can be kernel-based virtual machines (KVMs) and / or hardware-based virtual machines (QEMUs). Although storage device 110 is represented as a separate component from containers 104 and VMs 108, it can run as a container (e.g., container 104A) or within a VM (e.g., VM 108A). In some examples, implementing storage device 110 (e.g., object storage, block storage, etc.) as a container can be advantageous.

[0040] Figure 2 The edge devices described herein (e.g., Figure 1 An example architecture 200 connects edge device 100 to computing device 202 (e.g., a user computing device). Computing device 202 can be any type of computing device, including but not limited to laptops, desktop computers, etc. Edge device 204 ( Figure 1 An example of an edge device 100 may include a containerization engine 206 ( Figure 1 Example of containerization engine 102), hypervisor 208 ( Figure 1 Example of management program 106) and storage device 210 ( Figure 1 (Example of storage device 110).

[0041] Furthermore, as briefly mentioned above, edge device 100 may include API proxy 212 for managing RESTful API calls received from computing device 202. API calls may enter edge device 204 via network interface card (NIC) 214 within edge device 204. NIC 214 may be used to connect edge device 204 to computing device 202 via a local area network (e.g., LAN 216). API calls received by NIC 214 may be transmitted to an exposed endpoint (e.g., endpoint 218) that can implement a web server. The web server may transmit requests to API proxy 212, which may route requests to appropriate services (e.g., containerization engine 206, hypervisor 208, and / or storage device 210). The exposed endpoint / web server may also be configured to implement a lightweight console (e.g., a user interface displayed on computing device 202) for client use.

[0042] The lightweight console can run within a web browser (e.g., Mozilla Firefox) on a laptop, desktop computer, or other network-accessible device (e.g., connected to a local area network (LAN 216)) connected to the edge device 204 via a network (e.g., connected to a local area network (LAN 216)). The edge device 204 can expose endpoint 218 for console connectivity, and the web server can transmit data to the web browser on the computing device 202 via LAN 216.

[0043] Figure 3 It is a cloud infrastructure edge computing device according to at least one embodiment (e.g., edge device 300, respectively) Figure 1 Edge devices 100 and Figure 2 The example computer architecture diagram (example edge device 204) is shown. Edge device 300 can be thought of as a cloud integration service that extends some or all of the regular cloud capabilities to locations outside the cloud data center. This can be achieved via portable, hardened server nodes that provide cloud-like functionality in locations without WAN connectivity. This allows customers to move selected cloud workloads to remote locations and enable intensive data processing operations at the data ingestion point at the edge, close to their cloud infrastructure.

[0044] Edge device 300 may include any suitable number of services (e.g., one or more services 302). Each service may run locally on edge device 300 as a container (e.g., a Docker container). One or more services 302 may be communicatively connected via a substrate network 304, such that communication between services is encrypted (e.g., according to a security protocol such as MACsec). Each container may be assigned a substrate IP address (e.g., a static address) through which traffic can be addressed. In some embodiments, the security protocol (e.g., MACsec) is configured at provisioning time (e.g., before edge device 300 is shipped to a user). The system software of the edge device (including one or more services 302) may execute in a secure environment protected by boot security software (e.g., Trenchboot Secure Launch). User access to the secure environment and / or substrate network 304 may be restricted. To minimize the amount of resources used by these services, the service code may be compiled and stored on disk to reduce RAM space and lower the CPU load on edge device 300.

[0045] Some example services included in Service 302 (one or more) may include a UI console service, an identity control plane (CP) service, an identity data plane (DP) service, a compute application programming interface (API) service, a compute worker thread service, a virtual network (VN) API service, a block storage API service, a function-as-a-service service, an event service, an object storage management service (e.g., implementing a storage platform such as Ceph Storage (a Red Hat product), and a compute DP service (e.g., Figure 2 Examples of services include: Management Program 208 (VNDP service), Block Storage Management Service, Function as a Service API Service, Function as a Service Load Balancing (LB) Service, Function as a Service Process Thread Service, Distributed Data Storage Management Service (e.g., etcd3), Dynamic Host Configuration Protocol Service, Domain Name System Service, Network Time Protocol (NTP) Service, etc. Some example functionalities provided by these services are discussed below.

[0046] For example, a compute DP service can be configured (e.g., pre-configured and provisioned to edge device 300) to isolate one or more VMs 308 on the same hypervisor host. The compute DP service can utilize any suitable container engine (e.g., Docker containers, MicroContainers, etc.) to isolate one or more VMs 308 on the same hypervisor host from each other. The compute DP service can utilize any suitable hypervisor (e.g., Quick Emulator (QEMU), Kernel-based Virtual Machine (KVM), etc.) to provide virtual hardware emulation for one or more VMs 308. In some embodiments, one or more VNICs 306 are attached to a subnet of any suitable number of virtual networks (e.g., one or more private virtual networks (PVNs)) 305 and assigned private Internet Protocol (IP) addresses. A VM can have multiple VNICs from different VCNs and different subnets. The maximum number of VNICs can be limited by a predefined threshold (e.g., configuration data called “VM shape”, which defines the VNIC count, VNIC shape, etc., for each VM). In some embodiments, a predefined threshold is applied to each of the one or more VMs 308. VLANs can be used to isolate the subnets used by the one or more VNICs 306. In some embodiments, some or all of the one or more VNICs 306 can be assigned public and / or private IP addresses. Public IP addresses are addresses in the one or more networks 320, while private IP addresses refer to the IP addresses of the one or more PVNs 305.

[0047] In some embodiments, edge device 300 implements various networking functionalities via multiple services such as Network Address Translation (NAT), Dynamic Host Configuration Protocol (DHCP), Domain Name System (DNS), Network Time Protocol (NTP), metadata, and public API services. The metadata service can provide initialization data and other metadata to all(one or more) VMs 308. In some embodiments, the DHCP service assigns a private IP address to each of the one or more VNICs 306, each of the one or more VMs 308 having one or more VNICs. The DNS service can provide domain name resolution to the one or more VMs 308 on edge device 300. NTP can provide time synchronization to the one or more VMs 308. In some embodiments, the public IP service, performed as part of one or more services 302, enables VMs to access public APIs without assigning public IPs to VMs and without configuring a service gateway.

[0048] In some embodiments, at least one of the VMs 308 can implement block (or object) storage. In some embodiments, the hypervisor associated with the VM may include libraries that enable the hypervisor to use a distributed data storage platform (e.g., Ceph). This library may leverage protocols associated with the storage platform (e.g., RADOS block devices (RBDs)) to facilitate block-based data storage. The distributed data storage platform may be implemented on multiple VMs. In some embodiments, the distributed data storage platform supports snapshotting and replicating block volumes. VM images and VM block volumes may be Ceph block devices. In some embodiments, the VMs(s) implementing the distributed data storage platform will use system reserved resources (e.g., 8 CPU cores, some of the total number of CPUs available on the edge device 300). For example, to provision a boot volume, a block device image can be replicated to the block device's boot volume. The distributed data storage platform can use block devices comprising multiple nodes for redundancy. If some nodes fail, the block device can continue operating. In some embodiments, the distributed data storage platform (e.g., Ceph) automatically recovers block device data in the event of a few node failures. Block storage devices can be used to store images of any suitable deployable resources. For example, an image can be used to boot a VM. In some embodiments, the image may correspond to a specific VM shape (e.g., compute-intensive VM, GPU-optimized VM, storage VM, etc.).

[0049] The Compute API service can support the following operations: 1) VM startup and termination, 2) VM stop, start, and reboot, 3) listing VMs and / or obtaining information about a specific VM, 4) obtaining VM console history API, 5) obtaining VM snapshots, 6) attaching / detaching block volumes, and so on. In some embodiments, the Compute API service can be used to invoke other services (e.g., Compute DP service, Identity DP service for authentication and authorization, etc.).

[0050] Some of the functionalities of other services will be combined Figure 7 This will be discussed further. Generally, while each service may not be discussed in detail herein, the overall functionality provided by service(s)302 may include the functionality of cloud services provided by a remote cloud service provider. In some embodiments, edge device 300 may be associated with a predefined region and / or domain, such that some of service(s)302 can operate as if they were operating in a cloud computing environment, even though they actually operate as a single instance on one or more local devices (one or more edge devices), or as part of a distributed service that may not have public network access or has intermittent public network access to the cloud computing environment associated with the customer.

[0051] In some embodiments, edge device 300 may provide any suitable number of virtual networks (e.g., one or more private virtual networks 305) using computing, memory, and networking resources (e.g., one or more Virtual Network Interface Cards (VNICs) 306). A virtual network is a logical network operating on top of a physical underlying network. Using one or more services 302, one or more client resources or workloads (such as virtual machines (e.g., one or more virtual machines (VMs) 308) executing compute instances) may be deployed on these private virtual networks. Any suitable combination of one or more VMs 308 can perform functionality (e.g., compute instances, storage, etc.) that can be individually accessed through a virtual NIC (e.g., one or more virtual NICs 306). Each VM that is part of a PVN is associated with a VNIC that enables the VM (e.g., a compute instance) to become a member of a subnet of the PVN. The VNIC associated with a VM facilitates communication of packets or frames to and from the VM. A VNIC may be associated with a VM when the VM is created. One or more PVNs 305 may take various forms, including peer-to-peer networks, IP networks, etc. In some embodiments, the underlying network traffic of one or more services 302 may be encrypted and / or isolated from the network traffic of one or more VMs 308 running on the edge device 300 (e.g., by means of different PVNs or subnets).

[0052] Therefore, Edge Device 300 provides a collection of infrastructure and supplementary services that enable customers to build and run a wide range of applications (e.g., compute instances), services, and / or storage in highly available, physically local, and virtual-hosted environments. Customers do not manage or control the underlying physical resources provided by Edge Device 300, but they can control the scaling up or down of virtual machines (e.g., compute instances, virtual NICs, block or object storage devices, etc.), and the deployment of applications to these virtual machines. All workloads on Edge Device 300 can be split across different sets of CPUs (e.g., VMs and non-VMs). One set (e.g., non-VMs, such as workloads performed by one or more services 302) can utilize a subset of the CPU cores of Edge Device 300 (e.g., 8), while another set (e.g., VM workloads performed by one or more VMs 308) can utilize different subsets of the CPU cores.

[0053] Edge device 300 can communicatively connect to user equipment (e.g., NIC2 and / or NIC4) via one or more network interfaces (e.g., NIC2 and / or NIC4) and network 320. Figure 2The computing device 202 is used to interface with and / or manage one or more VMs 308. In some embodiments, a lightweight console may be provided at the user device via a web-based user interface that can be used to access and manage the edge device 300. In some embodiments, the console is a web-based application (e.g., one or more services 302) provided by the edge device 300.

[0054] Figure 3 A single edge device is described. However, it should be recognized that more than one edge device can be used as a distributed computing cluster.

[0055] Figure 4 It describes a scenario according to at least one embodiment including one or more edge computing devices (e.g., edge devices 402 and 404, each of which is...). Figure 3 A block diagram of a distributed computing cluster 400 (example of edge device 300).

[0056] Each edge device of the distributed computing cluster 400 can be connected via the underlying network 406 ( Figure 3 Example of a base network 304. In some embodiments, edge devices of the distributed computing cluster 400 (sometimes referred to as “edge computing nodes” or “edge nodes”) can be connected via a base network 406 using one or more switches (e.g., switches 408 and / or 410). In some embodiments, NIC1 and NIC5 may include specific connectors (e.g., RJ45 connectors), while NIC3 and NIC8 may include the same or different connectors (e.g., QSFP28 100 GbE connectors). In some embodiments, only one edge device in the distributed computing cluster 400 is connected to a network such as (one or more) network 420 (example of a base network 304). Figure 3 This is similar to a client network (e.g., one or more networks 320). Therefore, not only can traffic between services of edge devices be encrypted and isolated from other traffic of a given edge device, but traffic between distributed services operating across multiple edge devices can also be encrypted and isolated from other traffic of the computing cluster. In some embodiments, each edge device is pre-configured as a specific node in the distributed computing cluster 400. In other embodiments, the user can configure the number and topology of edge devices in the distributed computing cluster 400.

[0057] Trust establishment between edge devices in a cluster

[0058] As briefly discussed above, edge devices can be deployed as clusters in various locations with limited or no network connectivity to public networks or other networks that allow access to cloud services. In some examples, the lack of external network connectivity can be due to security restrictions that prevent connection to the cloud (e.g., protected private networks). For clusters of edge devices configured by a CSP, the lack of reliable network connectivity to the CSP may limit the provisioning and configuration of edge devices after deployment, or limit the provisioning and deployment of new edge devices to existing clusters. In particular, edge devices in an existing cluster may require mechanisms to establish trust (e.g., provide authentication and authorization) for new edge devices added to the cluster. If external network connectivity to the CSP or other cloud-based services (e.g., certificate authorities, etc.) is not available, existing edge devices in the cluster may not have the necessary chain of trust to establish secure network communication with newly added edge devices within the cluster.

[0059] To establish trust between edge devices without a network connection to the CSP, the CSP or its customers can manage clusters of edge devices as a group of associated edge devices. The CSP can configure each edge device in each group with appropriate information (e.g., credentials, encryption keys, certificates, etc.), allowing edge devices to verify each other within the cluster without communicating with the CSP or other services outside the cluster (or outside the protected private network in which the cluster operates, such as the CSP's customer's secure network). The CSP can implement an edge device cloud service to provide management of edge device clusters, including operations for configuring and provisioning edge devices before deployment to destination sites (e.g., customer facilities, security facilities, remote locations, etc.).

[0060] Each edge device can be associated with a cluster configured by the CSP. A cluster defines the association of edge devices that operate as a cluster of edge devices. For example, each cluster of edge devices can correspond to a cluster of edge devices deployed to a destination site. In some examples, one or more clusters can be associated with a cluster operated by a single customer of the CSP. Each cluster can be associated with a cluster master encryption key, which can be configured on each device associated with that cluster and used to encrypt / decrypt messages used for authentication and / or authorization of each edge device within the cluster. Because this cluster master encryption key can be a shared key, symmetric encryption methods can be used within the cluster of edge devices to establish trust between them.

[0061] The techniques described in this paper offer numerous advantages over typical approaches to deploying computing devices in clusters in remote locations. For example, managing edge devices as part of a cluster allows for the secure integration of new edge devices into the cluster without connecting to the cloud via public or other networks, thereby maintaining the appropriate security posture of the cluster's protected network and allowing edge devices to verify the authenticity of added devices without relying on external services. As another example, pre-configuring edge devices with sufficient information to perform edge device authentication and authorization reduces manual configuration of field devices, which in turn reduces the number of operations required to integrate devices into an existing cluster and ensures that the existing cluster can provide cloud services during the addition of new devices.

[0062] Figure 5 This is a block diagram depicting a distributed computing system 500 according to some embodiments, the distributed computing system including a distributed computing cluster 502 of one or more edge devices managed by a CSP 512 as part of a cluster 516. According to at least one embodiment, the distributed computing cluster 502 may be similar to Figure 4 The distributed computing cluster 400, while the edge devices 504-510 can be... Figure 5 An example of edge device 300. Distributed computing cluster 502 may include one or more edge devices, including edge device 1 504, edge device 2 506, edge device 3 508, up to edge device N 510. Edge devices 504-510 may be connected together via a basic network within distributed computing cluster 502, and connected to additional networks (e.g., ...). Figure 4 (One or more networks 420). For example, a distributed computing cluster 502 may be installed at a secure customer facility and connected to the customer network.

[0063] CSP 512 can provide configuration, provisioning, and management of edge devices 504-510 before they are deployed to the destination site. CSP 512 can provide an edge device cloud service 514 configured to perform operations to configure edge devices 504-510. For example, edge device cloud service 514 can provision encryption keys, certificates, identifiers, and other information to edge devices 504-510. Configuration operations can be performed on edge devices 504-510 before they are shipped or delivered to the destination site. Configuration operations may also include associating edge devices 504-510 with a cluster 516. Cluster 516 can be a logical abstraction of edge devices 504-510 associated within a distributed computing cluster. In some embodiments, cluster 516 can represent a cluster of more than one edge device. Edge device cloud service 514 can also be configured to maintain configuration information associated with cluster 516. For example, the edge device cloud service 514 can maintain cluster encryption keys and public keys for edge devices in the cluster for other configuration operations of the edge devices associated with the cluster (e.g., configuring new edge devices for an existing cluster).

[0064] Distributed computing cluster 502 can operate at customer sites, remote locations, or other sites lacking network connectivity to CSP 512. For example, distributed computing cluster 502 can operate in remote facilities (e.g., oil drilling facilities) that do not have network access to the public internet. In another example, distributed computing cluster 502 can operate in a secure network (e.g., a government or defense facility with restricted or otherwise secure network operations), for which connectivity to external resources may be restricted or prohibited. Therefore, configuring edge devices 504-510 by edge device cloud service 514 may only be possible before deploying edge devices 504-510 to the site hosting distributed computing cluster 502.

[0065] Figure 6 This is a block diagram depicting a cluster of edge devices supplied by a CSP 612 in a distributed computing system 600 according to some embodiments. The CSP 612 may be as described above. Figure 5 The example described is CSP 512, while edge device cloud service 614 could be... Figure 5 The example of edge device cloud service 514. Edge devices 604-610 can each be examples of the edge devices described herein, including Figure 3 Edge devices 300.

[0066] During the provisioning of edge devices as part of a fleet, the edge devices can connect to the edge device cloud service 614 via one or more networks 622. The one or more networks 622 can be any suitable network, including public networks (e.g., the Internet), private networks, cellular networks, local area networks, or any other such network or combinations thereof. The components used in such a system may depend at least in part on the chosen environment and / or the type of network. The protocols and components used for communication via such networks are well known and will not be discussed in detail herein. Communication via the one or more networks 622 can be enabled via wired or wireless connections and combinations thereof.

[0067] Configuring edge devices 604-610 can occur at a facility where edge devices 604-610 are physically under the supervision of CSP 612 and / or at a facility associated with CSP 612 and accessible via one or more networks 622. For example, CSP 612 may use the facility to perform configuration and deploy software to the edge devices before delivery to the customer. Once the cluster has been configured, the associated edge devices can be shipped to the destination site (e.g., the customer facility). Once shipped to the destination site, the edge devices may no longer be able to connect to the edge device cloud service 614 for further configuration operations.

[0068] Configuring edge devices can include associating edge devices with a cluster. Figure 6 In the example depicted, edge devices 1 604 and 2 606 may be associated with cluster 1 624, while edge devices 3 608 and 4 610 may be associated with cluster 2 626. In some embodiments, clusters 1 624 and 2 626 may include more than two edge devices. The number of edge devices in a cluster may not be fixed. For example, additional edge devices may be associated with the cluster at a future time to support the ability to expand the distributed computing cluster capabilities of the edge devices (e.g., compute, storage, etc.). Similarly, deployed edge devices associated with a cluster may fail or otherwise need to be replaced. Replacement edge devices may be configured by the edge device cloud service 614 before being shipped to the destination site.

[0069] The edge device cloud service 614 can configure each edge device using a master encryption key associated with each cluster. For example, edge device 1 604 and edge device 2 606 can be supplied with cluster 1 master key 630. The master encryption key can be any suitable secret key used in a symmetric keying method, including an Advanced Encryption Standard (AES) 128-bit key, an AES 256-bit key, etc. Each edge device can include a key repository for storing keys and other secrets. The key repository can be a Trusted Platform Module (TPM), a Hardware Security Module (HSM), or a similar component for managing keys and other secrets.

[0070] Each edge device can have a corresponding public key, which can be generated as part of a public / private key pair. For example, edge device 1 604 can have public key 632, edge device 2 606 can have public key 634, edge device 3 608 can have public key 636, and edge device 4 610 can have public key 638. The public keys can be provided to the edge device cloud service 614. When an edge device is associated with a cluster, the public key of each other edge device associated with the same cluster can be supplied to the edge device. For example, during supplying, the edge device cloud service 614 can provide the public key 634 from edge device 2 606 to edge device 1 604, since both are associated with cluster 1 624. Similarly, the edge device cloud service 614 can provide the public key 632 from edge device 1 604 to edge device 2 606. Likewise, edge device 3 608 and edge device 4 610 can be provided with public key 638 and public key 636, respectively. Public keys can also be stored in a key repository on each edge device.

[0071] Edge device cloud service 614 may store configuration information in one or more data repositories. For example, CSP 612 may include a cluster data repository 616. Cluster data repository 616 may be configured to generate and / or maintain a master key 618 (e.g., a copy of cluster 1 master key 630 and cluster 2 master key 640) for a cluster managed by edge device cloud service 614. Cluster data repository 616 may also be configured to maintain a public key 620. Public key 620 may include the public keys of all edge devices configured by edge device cloud service 614. In some embodiments, edge device cloud service 614 may retrieve stored keys from cluster data repository 616 for use when configuring additional edge devices. For example, if a new edge device is to be added to cluster 1 624, edge device cloud service 614 may retrieve cluster 1 master key 630, public key 632, and public key 634 from cluster data repository 616 to provide to the new edge device. Furthermore, when configuring a new edge device, the edge device cloud service 614 can obtain the new edge device's public key and store it together with the public key 620 in the cluster data repository 616. In this way, an existing cluster can be expanded with additional edge devices.

[0072] In some embodiments, edge devices can be delivered to the destination site without being configured by the edge device cloud service 614 and without being associated with a cluster. For example, some customers may want to configure a cluster of edge devices without CSP 612 maintaining configuration information. In these cases, the customer can be provided with a master key for use when configuring the edge devices, and this master key will function in other ways like the cluster master encryption key described above.

[0073] Figure 7 This is a block diagram depicting the configuration of an edge device 702 with an encryption key in a distributed computing system 700 as part of associating an edge device with a cluster, according to some embodiments. Edge device 702 may be an example of other edge devices described herein, including... Figure 3 The configuration of edge device 300 and edge device 702 can be performed by edge device cloud service 714 of CSP 712. CSP 712 can be the one mentioned above. Figure 6 The example described is CSP 612, while edge device cloud service 714 could be... Figure 6 An example of an edge device cloud service 714. The edge device cloud service 714 can communicate with the edge device 702 via one or more networks 722, which can be... Figure 6 Examples of one or more networks 622.

[0074] Edge device 702 may be associated with cluster 1. Configuring edge device 702 may include providing edge device 702 with cluster 1 master key 730. Cluster 1 master key 730 may be stored by edge device cloud service 714 in cluster data repository 716 as part of master key 718 maintained by edge device cloud service 714. Configuring edge device 702 may also include providing edge device 702 with public key 736. Public key 736 may include public keys of other edge devices associated with cluster 1. In some embodiments, edge device 702 may be the only edge device initially configured for cluster 1. For example, in a small-scale deployment, a cluster of a single edge device may be sufficient to provide cloud computing functionality. In this case, since no other edge devices are associated with cluster 1, public key 736 may not need to be provided to edge device 702 (because they do not yet exist). As the resources required by the edge device cluster expand, additional edge devices may be added to cluster 1 to increase the size and computing power of the cluster.

[0075] Edge device 702 may also include a public / private key pair, comprising a public key 732 and a private key 734. The public key 732 and private key 734 can be generated using any suitable technology for generating the encryption key for public key encryption. Edge device 702 can provide the public key 732 to edge device cloud service 714. Edge device cloud service 714 can then store the public key 732 as part of public key 720 in cluster data repository 716.

[0076] Edge device 702 can store encryption keys, certificates, and other secrets in key repository 724. Key repository 724 can be implemented as a distinctly different chip, TPM, HSM, integrated circuit platform, or other hardware, firmware, and / or software to provide secure storage and security management of the stored secrets, including encryption keys such as public key 732, private key 734, cluster 1 master key 730, and public key 736. Key repository 724 can be encrypted separately by edge device 702.

[0077] In some embodiments, in addition to providing an encryption key to edge device 702, edge device cloud service 714 may also provide one or more cluster policies 726. Cluster policy 726 may include rules, configuration settings, and other information specifying the behavior of edge devices within the cluster. For example, cluster policy 726 may include rules (e.g., port, link settings, communication protocols, etc.) for networking protocols that edge device 702 can use when connecting to and communicating with other edge devices in the cluster. Cluster policy 726 may also specify parameters for authorization and authentication used to establish trust between edge device 702 and other edge devices (e.g., new edge devices added to the cluster). For example, cluster policy 726 may specify parameters (e.g., timing, protocol, etc.) for authorization / authentication messages(s) used to establish trust when a new edge device is installed into the cluster of an existing cluster. The following is about... Figures 8-10 Provide additional details on the trust-building process.

[0078] Figure 8 This is a block diagram depicting communication between two edge devices in a distributed computing cluster 800 according to some embodiments. The distributed computing cluster may be an example of other distributed computing clusters described herein, including... Figure 5 The distributed computing cluster 502. Edge device 1 802 and edge device 2 804 may be examples of other edge devices described herein, including Figure 7 Edge devices 702 and Figure 3 Edge devices 300.

[0079] Edge device 1 802 and edge device 2 804 may be associated with a cluster (e.g., cluster 1). Edge device 2 804 may be a new edge device of the distributed computing cluster 800. Therefore, edge device 2 804 may be configured with the public key (e.g., public key 814 including public key 806) of an existing edge device (e.g., edge device 1 802) of the distributed computing cluster 800, while edge device 1 802 may be configured with a public key (e.g., public key 812) that does not include the public key 2 816 of edge device 2 804. Because edge device 1 802 does not have the public key 2 816 of edge device 2 804 (or any relevant certificates or other information proving the identity of edge device 2 804), edge device 1 802 cannot verify messages sent from edge device 2 804 using private key 2818. For example, if edge device 2 804 sends a message encrypted (or cryptographically signed) with private key 2 818 to edge device 1 802, edge device 1 802 will not be able to decrypt the message (or verify the signature) without public key 2 816.

[0080] Because a distributed computing cluster 800 may not have a connection to a CSP (e.g., Figure 7 Edge device 2 804 can securely and reliably provide its public key 2 816 to edge device 1 802 via external network access (using CSP 712) to obtain public key 732. Edge device 2 804 can send an encrypted broadcast message 824 to edge device 1 802. Edge device 2 804 can encrypt the encrypted broadcast message 824 using cluster 1 master key 810. Upon receiving the encrypted broadcast message 824, edge device 1 802 can decrypt it using a copy of its cluster 1 master key 810. If decryption is successful, edge device 2 804 is verified as a legitimate device associated with cluster 1. Edge device 1 802 can then update its public key 812 storage using public key 2 816. In some embodiments, the encrypted broadcast message 824 may also include a network address corresponding to edge device 2 804. This network address can be used as an identifier for edge device 2 804 within the distributed computing cluster 800. After successfully decrypting the encrypted broadcast message 824, edge device 1 802 can also store the network address of edge device 2804 and associate that network address with a trusted device within the distributed computing cluster 800.

[0081] The above description can be applied to multiple edge devices in a cluster. For example, multiple new edge devices can be added simultaneously to a distributed computing cluster 800 as part of cluster 1. Each of these new edge devices can be configured with the public keys of existing edge devices in the cluster (e.g., public key 1 806, public key 2 816). Each new edge device can send an encrypted broadcast message to every other edge device in the distributed computing cluster 800. Once received, all edge devices can decrypt the encrypted broadcast message from each other edge device using the shared cluster 1 master key 810. Successful decryption of the received broadcast message authenticates the new edge device, and the shared public key can be added to the stored public keys of the edge device.

[0082] Once edge device 1 802 updates its public key 812 with public key 2 816, edge device 1 802 and edge device 2 804 can use session tokens to establish an authorized and authenticated communication channel. To obtain a session token, edge device 1 802 can send a session token request 828 to edge device 2 804. The session token request 828 can be cryptographically signed by edge device 1 802 using its private key 1 808. Upon receiving the session token request 828, edge device 2 804 can verify the signature of the session token request 828 using a copy of public key 1 806 stored along with public key 814. If the signature verification is successful, edge device 2 804 can trust that the session token request 828 was validly sent from edge device 1 802. In response, edge device 2 804 can send a session token 830 to edge device 1 802. Subsequent requests from edge device 1 802 (e.g., API requests) may include session token 830, which proves the valid authentication of edge device 1 802 as presented when parsing session token request 828.

[0083] Figure 9 This is an example process 900 for the provisioning and deployment of two edge devices according to some embodiments. CSP 908 can use an edge device cloud service to provision edge device 1 902 and edge device 2 904 to CSP 908's customer 906. Edge device 1 902 and edge device 2 904 can be similar to those described above. Figure 8 Edge devices 1 802 and 2 804 are described, while CSP 908 can be similar. Figure 7 CSP 712.

[0084] At step 910, customer 906 may request edge device 1 902. This request may be evaluated by CSP 908, which may approve the request at step 912. Once approved, CSP 908 may provision edge device 1 902 at step 914. Provisioning edge device 1 902 may include associating edge device 1 902 with a cluster managed by CSP 908, providing the public keys of other edge devices associated with the cluster (if present), and providing the cluster master key. These encryption keys may be stored in a keystore of edge device 1 902. CSP 908 may also perform other operations to configure edge device 1 902 to operate according to customer 906's requirements. For example, edge device 1 902 may be provisioned with appropriate software to provide desired cloud services at an edge location. Once configured, CSP 908 may ship edge device 1 902 to customer 906.

[0085] At step 918, customer 906 may request edge device 2 904. This request may specify adding edge device 2 904 to the same cluster as edge device 1 902. Similar to a request for edge device 1, CSP 908 may evaluate the request and approve it at step 920. Once approved, CSP 908 may provision edge device 2 904 at step 922. Provisioning edge device 2 904 may include associating edge device 2 904 with the cluster, providing the public keys of other edge devices associated with the cluster (including the public key of edge device 1 902), and providing the cluster master key. These encryption keys may be stored in a keystore of edge device 2 904. CSP 908 may also perform other operations to configure edge device 2 904 to operate according to customer 906's request. Once configured, CSP 908 may ship edge device 2 904 to customer 906.

[0086] Once edge device 1 902 and edge device 2 904 are installed at the destination site (e.g., as part of a distributed computing cluster), each edge device can establish trust with the other. At step 926, edge device 2 904 can send an encrypted broadcast message to edge device 1 902 (e.g., ...). Figure 8 The encrypted broadcast message (824) may include the public key of edge device 2 904. The encrypted broadcast message can be encrypted using the cluster master key. At step 928, edge device 1 902 can decrypt the encrypted broadcast message using the cluster master key stored at edge device 1 902. If decryption is successful, edge device 1 902 can add the public key of edge device 2 904 to its key store.

[0087] Additionally or alternatively, edge device 1 902 may send an encrypted broadcast message to edge device 2 904 at step 930. The encrypted broadcast message may include the public key of edge device 1 902. Edge device 2 904 may decrypt the encrypted broadcast message using the cluster master key. If decryption is successful, edge device 2 904 may add the public key of edge device 1 902 to its keystore (if necessary). In some embodiments, all edge devices in the same cluster and connected together in the network within the cluster may send encrypted broadcast messages to all other edge devices in the cluster. The conditions for sending encrypted broadcast messages (e.g., when a new device is detected within the cluster network, during the initial startup of a new edge device, etc.) may be determined by cluster policies (e.g., Figure 7 As specified in the cluster strategy (726).

[0088] At step 934, edge device 1 902 can request a session token from edge device 2 904. The request for the session token can be signed using the public key of edge device 2 904, which is received as part of an encrypted broadcast message. At step 936, edge device 2 904 can verify the request for the session token by verifying the signature using its private key. If the signature is correctly verified, edge device 2 904 can return the requested session token to edge device 1 902. In some embodiments, steps similar to 934-938 can be performed by edge device 2 904 requesting a session token from edge device 1 902.

[0089] At step 940, edge device 1 902 can establish an authenticated communication channel using the received session token. Edge device 1 902 can send a request with the token (e.g., an API request). At step 942, edge device 2 904 can verify the received session token by comparing it with the token provided at step 938. If the tokens match, then edge device 2 904 can respond to the request at step 944 and provide a return to edge device 1 902.

[0090] Figure 10 This is an example process 1000 for establishing trust between two edge devices in a distributed computing system, according to some embodiments. The edge devices can be located in a distributed computing cluster (e.g., Figure 8 It operates in a distributed computing cluster (800) and can initially be provided by edge devices cloud services (e.g., Figure 6 Edge device cloud services (614) provision. Edge devices can be examples of other edges described herein, including Figure 9 Edge devices 1 902 and 2 904. Process 1000 is illustrated as a logic flow graph, where each operation represents a sequence of operations that can be implemented using hardware, computer instructions, or a combination thereof. In the context of computer instructions, an operation represents a computer-executable instruction stored on one or more computer-readable storage media that performs the operation when executed by one or more processors. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc., that perform a specific function or implement a specific data type. The order in which operations are described is not intended to be construed as limiting, and any number of described operations may be omitted or combined in any order and / or in parallel to implement the process.

[0091] Some, any, or all of process 1000 (or any other process described herein, or variations and / or combinations thereof) may be executed under the control of one or more computer systems configured with executable instructions, and may be implemented by hardware or a combination thereof as code (e.g., executable instructions, one or more computer programs, or one or more applications) that executes jointly on one or more processors. The code may be stored on a computer-readable storage medium (e.g., in the form of a computer program comprising multiple instructions executable by one or more processors). The computer-readable storage medium may be non-transitory.

[0092] At box 1002, the edge device cloud service can associate a first cloud computing edge device with a cluster of cloud computing edge devices. Associating the first cloud computing edge device may include obtaining a first public encryption key of the first cloud computing edge device (e.g., Figure 8 The first public encryption key may be part of a public / private key pair corresponding to the first cloud computing edge device. The first public encryption key may be stored together with other public encryption keys of the cluster managed by the edge device cloud service in a cluster data repository (e.g., ...). Figure 6 The edge device cloud service can also maintain a master encryption key associated with the cluster (e.g., as part of the cluster data repository 616). For example, each cluster managed by the edge device cloud service can have an AES-256 encryption key, which can be provided to each edge device associated with the cluster during edge device configuration. The edge device cloud service can store the master encryption key in the cluster data repository (e.g., as part of the cluster data repository 616). Figure 6 Master key 618).

[0093] At box 1004, the edge device cloud service can provide a master encryption key (e.g., Figure 8 The first cloud computing edge device (with a master key 810 for cluster 1) is provided with a master encryption key. Providing the master encryption key to the first cloud computing edge device may include providing the master encryption key to the first cloud computing edge device. The first cloud computing edge device may store the master encryption key in a first key repository, which in some embodiments may be a TPM (Trusted Product Management System).

[0094] At box 1006, the edge device cloud service can associate a second cloud computing edge device with the cluster. Associating the second cloud computing edge device may include obtaining a second public encryption key for the second cloud computing edge device (e.g., Figure 8 The second public encryption key can be part of a second public / private key pair corresponding to the second cloud computing edge device. The edge device cloud service can maintain the second public encryption key in the cluster data repository.

[0095] At block 1008, the edge device cloud service can provide a second cloud computing edge device with a master encryption key and a first public encryption key. In some embodiments, the first cloud computing edge device can associate with the cluster before the second cloud computing edge device, so the second public key is not yet available for the edge device cloud service to provide to the first cloud computing edge device. The second cloud computing edge device can store the master encryption key and the first public encryption key in a second key repository on the second cloud computing edge device. In some embodiments, the second key repository can be a TPM (Telematics Management Device).

[0096] At box 1010, the first cloud computing edge device can receive encrypted message data from the second cloud computing edge device. The encrypted message data can be an encrypted broadcast message (e.g., Figure 8 This is part of an encrypted broadcast message (824). The encrypted message data may include a second public encryption key. The encrypted message data may be generated by the second cloud computing edge device using a master encryption key. For example, the second cloud computing edge device may encrypt the second public encryption key using the master encryption key. In some embodiments, the second cloud computing edge device may perform a cryptographic signature on the message data including the second public encryption key.

[0097] At block 1012, the first cloud edge device can decrypt the encrypted message data using the master encryption key stored in the first key repository. Because the first and second cloud edge devices are supplied as part of the same cluster, they should store the same shared master encryption key for that cluster. If decryption is successful (e.g., the master encryption key matches at both edge devices, indicating a real device within the cluster), the first cloud edge device can update the first key repository with the second public encryption key at block 1014. The first cloud edge device can then have a trusted, verified version of the second cloud edge device's public key without needing to connect to a cloud service or other external network entity (e.g., a certificate authority) to obtain authentication information. In some embodiments, the encrypted message data may also include a network address corresponding to the second cloud edge device. The first cloud edge device can then verify the request by comparing at least the network address received in the encrypted message data with the network address of the second cloud edge device associated with the communication channel through which it receives the encrypted message data.

[0098] In some embodiments, the edge device cloud service can associate a third, fourth, or more cloud edge devices with the cluster. Similar to the first and second cloud edge devices, associating the third cloud edge device may include obtaining a third public encryption key and storing it along with the cluster's public key. The edge device cloud service can then supply the third cloud edge device with the master encryption key and the first public encryption key of the first cloud edge device and the second public encryption key of the second cloud edge device.

[0099] In some embodiments, a first cloud computing edge device may receive a request for a session token from a second cloud computing edge device. This request may include a cryptographic signature generated using a second private encryption key, which may be stored in a second key repository on the second cloud computing edge device. Upon receiving the request, the first cloud computing edge device may verify the signature using a second public encryption key stored in a first key repository. Because the second public encryption key and the second private encryption key are part of a public / private key pair, each key can be used as part of a public-key encryption method to generate and verify the cryptographic signature. If the signature is successfully verified, the first cloud computing edge device may send a session token to the second cloud computing edge device.

[0100] In some embodiments, a first cloud computing edge device may send an application programming interface (API) request to a second cloud computing edge device. This API request may include a session token previously received from the second cloud computing edge device. In response, the first cloud computing edge device may receive an indication from the second cloud computing edge device that the API request has been successfully completed. This indication may be generated at least in part based on the second cloud computing edge device's successful verification of the session token.

[0101] In some embodiments, a first cloud computing edge device may receive additional encrypted message data from a second cloud computing edge device. This additional encrypted message data may include an updated public encryption key. For example, the second cloud computing edge device may generate a new public / private key pair for use in secure communication within a distributed computing cluster. The additional encrypted message data can be encrypted using a master encryption key. The first cloud computing edge device can decrypt the additional encrypted message data using the master encryption key and then update the first key repository with the updated public encryption key.

[0102] In some embodiments, edge device cloud services can be implemented using one or more cluster strategies (e.g., Figure 7The cluster policy (726) is provided to either the first or second cloud computing edge device. The cluster policy can specify multiple permissible actions for the cloud computing edge device cluster. For example, the cluster policy can include rules (e.g., port, link settings, communication protocols, etc.) for networking protocols that the first cloud computing edge device can use when connecting to and communicating with the second cloud computing edge device. The cluster policy can also specify authorization and authentication parameters used to establish trust between the first and second cloud computing edge devices. For example, the cluster policy can specify parameters (e.g., timing, protocol, etc.) for broadcast messages used to establish trust when a new edge device is installed into an existing cluster. The first cloud computing edge device can receive requests from the second cloud computing edge device that can be used to perform actions. In response, if the action is one of the permissible actions, the first cloud computing edge device can perform that action.

[0103] Example Infrastructure as a Service architecture

[0104] As noted above, Infrastructure as a Service (IaaS) is a specific type of cloud computing. IaaS can be configured to provide virtualized computing resources over a public network (e.g., the Internet). In the IaaS model, cloud providers can host infrastructure components (e.g., servers, storage devices, network nodes (e.g., hardware), deployment software, platform virtualization (e.g., hypervisor layer), etc.). In some cases, IaaS providers can also provision various services to accompany these infrastructure components (example services include billing software, monitoring software, logging software, load balancing software, and clustering software, etc.). Therefore, because these services may be policy-driven, IaaS users can implement policies to drive load balancing to maintain application availability and performance.

[0105] In some instances, IaaS customers can access resources and services over a wide area network (WAN) such as the internet and can use the cloud provider's services to install the remaining elements of the application stack. For example, a user can log in to the IaaS platform to create virtual machines (VMs), install an operating system (OS) on each VM, deploy middleware such as databases, create buckets for workloads and backups, and even install enterprise software into the VM. The customer can then use the provider's services to perform various functions, including balancing network traffic, troubleshooting application issues, monitoring performance, and managing disaster recovery.

[0106] In most cases, cloud computing models may require the involvement of cloud providers. Cloud providers can, but are not necessarily, third-party providers specializing in (e.g., provisioning, renting, selling) IaaS services. Entities may also choose to deploy private clouds, thus becoming their own infrastructure service providers.

[0107] In some examples, IaaS deployment is the process of placing a new application or a new version of an application onto a prepared application server, etc. It may also include the processing of server preparation (e.g., installation libraries, daemons, etc.). This is typically managed by the cloud provider, below the hypervisor layer (e.g., servers, storage devices, network hardware, and virtualization). Therefore, the customer can be responsible for processing (OS), middleware, and / or application deployment (e.g., on self-service virtual machines, etc., which can be started on demand).

[0108] In some examples, IaaS provisioning can refer to acquiring computers or virtual hosts for use, or even installing necessary libraries or services on them. In most cases, deployment does not include provisioning, and provisioning may need to be performed first.

[0109] In some cases, IaaS provisioning presents two distinct challenges. First, there's the initial challenge of provisioning the initial infrastructure set before anything is operational. Second, once everything is provisioned, there's the challenge of evolving the existing infrastructure (e.g., adding new services, changing services, removing services, etc.). In some cases, both challenges can be addressed by enabling configuration that declaratively defines the infrastructure. In other words, the infrastructure (e.g., which components are needed and how they interact) can be defined by one or more configuration files. Therefore, the overall topology of the infrastructure (e.g., which resources depend on which resources and how they work together) can be described declaratively. In some instances, once the topology is defined, workflows for creating and / or managing the different components described in the configuration files can be generated.

[0110] In some examples, the infrastructure can have many interconnected components. For example, there may be one or more Virtual Private Clouds (VPCs) (e.g., potential on-demand pools of configurable and / or shared computing resources), also known as the core network. In some examples, one or more inbound / outbound traffic group rules may also be supplied to define how inbound / outbound traffic to the network and one or more virtual machines (VMs) will be configured. Other infrastructure elements, such as load balancers, databases, etc., may also be supplied. The infrastructure can evolve incrementally as more and / or additional infrastructure elements are desired.

[0111] In some instances, continuous deployment techniques can be used to enable the deployment of infrastructure code across various virtual computing environments. Furthermore, the described techniques enable infrastructure management within these environments. In some examples, service teams may write code that they expect to deploy to one or more, but often many, different production environments (e.g., across various geographical locations, sometimes spanning the entire world). However, in some examples, it may be necessary to first set up the infrastructure on which the code will be deployed. In some instances, provisioning can be done manually, resources can be provisioned using provisioning tools, and / or once the infrastructure is provisioned, deployment tools can be used to deploy the code.

[0112] Figure 11 This is a block diagram 1100 illustrating an example pattern of an IaaS architecture according to at least one embodiment. Service operator 1102 may communicatively couple to secure host lease 1104, which may include a virtual cloud network (VCN) 1106 and a secure host subnet 1108. In some examples, service provider 1102 may use one or more client computing devices (which may be portable handheld devices (e.g., iPhone®, cellular phone, iPad®, computing tablet, personal digital assistant (PDA)) or wearable devices (e.g., Google Glass® head-mounted display), running software (such as Microsoft Windows Mobile®) and / or various mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 8, Palm OS, etc.), and supporting the Internet, email, short message service (SMS), Blackberry®, or other communication protocols). Alternatively, client computing devices may be general-purpose personal computers, including, for example, personal computers and / or laptops running various versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems. Client computing devices may be workstation computers running any of the various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, such as Google Chrome OS). Alternatively or additionally, client computing devices may be any other electronic device, such as a thin client computer, an Internet-enabled gaming system (e.g., a Microsoft Xbox game console with or without Kinect® gesture input), and / or capable of accessing a VCN. A personal messaging device that communicates with networks such as 1106 and / or the Internet.

[0113] VCN 1106 may include a local peering gateway (LPG) 1110, which may be communicatively coupled to SSH VCN 1112 via LPG 1110 contained in Secure Shell (SSH) VCN 1112. SSH VCN 1112 may include an SSH subnet 1114, and SSH VCN 1112 may be communicatively coupled to control plane VCN 1116 via LPG 1110 contained in control plane VCN 1116. Furthermore, SSH VCN 1112 may be communicatively coupled to data plane VCN 1118 via LPG 1110. Control plane VCN 1116 and data plane VCN 1118 may be contained in a service lease 1119 that may be owned and / or operated by an IaaS provider.

[0114] The control plane VCN 1116 may include a control plane demilitarized zone (DMZ) layer 1120 that acts as a peripheral network (e.g., part of a corporate network between an internal and external network). DMZ-based servers can assume limited liability and help control security breaches. Furthermore, the DMZ layer 1120 may include one or more load balancer (LB) subnets 1122, a control plane application layer 1124 that may include one or more application (app) subnets 1126, and a control plane data layer 1128 that may include one or more database (DB) subnets 1130 (e.g., one or more front-end DB subnets and / or one or more back-end DB subnets). One or more LB subnets 1122 contained in the control plane DMZ layer 1120 may be communicatively coupled to one or more application subnets 1126 contained in the control plane application layer 1124 and an Internet gateway 1134 that may be contained in the control plane VCN 1116. The application subnets 1126 may be communicatively coupled to one or more DB subnets 1130 contained in the control plane data layer 1128, as well as a service gateway 1136 and a Network Address Translation (NAT) gateway 1138. The control plane VCN 1116 may include the service gateway 1136 and the NAT gateway 1138.

[0115] The control plane VCN 1116 may include a data plane mirror application layer 1140, which may include one or more application subnets 1126. The one or more application subnets 1126 included in the data plane mirror application layer 1140 may include a virtual network interface controller (VNIC) 1142 capable of executing a compute instance 1144. The compute instance 1144 may communicatively couple the one or more application subnets 1126 of the data plane mirror application layer 1140 to the one or more application subnets 1126 that may be included in the data plane application layer 1146.

[0116] Data plane VCN 1118 may include data plane application layer 1146, data plane DMZ layer 1148, and data plane data layer 1150. Data plane DMZ layer 1148 may include one or more LB subnets 1122 communicatively coupled to one or more application subnets 1126 of data plane application layer 1146 and Internet gateway 1134 of data plane VCN 1118. One or more application subnets 1126 communicatively coupled to service gateway 1136 and NAT gateway 1138 of data plane VCN 1118. Data plane data layer 1150 may also include one or more DB subnets 1130 communicatively coupled to one or more application subnets 1126 of data plane application layer 1146.

[0117] The Internet gateway 1134 of control plane VCN 1116 and data plane VCN 1118 can be communicatively coupled to metadata management service 1152, which can be communicatively coupled to public Internet 1154. Public Internet 1154 can be communicatively coupled to NAT gateway 1138 of control plane VCN 1116 and data plane VCN 1118. Service gateway 1136 of control plane VCN 1116 and data plane VCN 1118 can be communicatively coupled to cloud service 1156.

[0118] In some examples, service gateway 1136, either control plane VCN 1116 or data plane VCN 1118, can make application programming interface (API) calls to cloud service 1156 without traversing the public internet 1154. API calls from service gateway 1136 to cloud service 1156 can be unidirectional: service gateway 1136 can make API calls to cloud service 1156, and cloud service 1156 can send requested data to service gateway 1136. However, cloud service 1156 may not initiate API calls to service gateway 1136.

[0119] In some examples, secure host lease 1104 can be directly connected to service lease 1119, which would otherwise be isolated. Secure host subnet 1108 can communicate with SSH subnet 1114 via LPG 1110, which enables bidirectional communication between otherwise isolated systems. Connecting secure host subnet 1108 to SSH subnet 1114 allows secure host subnet 1108 to access other entities within service lease 1119.

[0120] Control plane VCN 1116 may allow users of service lease 1119 to configure or otherwise provision desired resources. Desired resources provisioned in control plane VCN 1116 may be deployed or otherwise used in data plane VCN 1118. In some examples, control plane VCN 1116 may be isolated from data plane VCN 1118, and the data plane mirror application layer 1140 of control plane VCN 1116 may communicate with the data plane application layer 1146 of data plane VCN 1118 via VNIC 1142, which may be included in both the data plane mirror application layer 1140 and the data plane application layer 1146.

[0121] In some examples, users or clients of the system can make requests, such as create, read, update, or delete (CRUD) operations, via the public internet 1154, which can transmit requests to the metadata management service 1152. The metadata management service 1152 can transmit requests to the control plane VCN 1116 via internet gateway 1134. Requests can be received by one or more LB subnets 1122 contained in the control plane DMZ layer 1120. The LB subnets 1122 can determine that the request is valid, and in response to this determination, they can transmit the request to one or more application subnets 1126 contained in the control plane application layer 1124. If the request is validated and requires a call to the public internet 1154, the call to the public internet 1154 can be transmitted to a NAT gateway 1138 that can make calls to the public internet 1154. The request may expect the storage to be located in one or more DB subnets 1130.

[0122] In some examples, the data plane mirroring application layer 1140 can facilitate direct communication between the control plane VCN 1116 and the data plane VCN 1118. For example, it may be desirable to apply configuration changes, updates, or other appropriate modifications to resources contained in the data plane VCN 1118. Through VNIC 1142, the control plane VCN 1116 can communicate directly with the resources contained in the data plane VCN 1118, and thus can perform configuration changes, updates, or other appropriate modifications to these resources.

[0123] In some embodiments, the control plane VCN 1116 and data plane VCN 1118 may be included in service lease 1119. In this case, the system's users or customers may not own or operate the control plane VCN 1116 or data plane VCN 1118. Alternatively, the IaaS provider may own or operate both the control plane VCN 1116 and data plane VCN 1118, and both planes may be included in service lease 1119. This embodiment can enable the isolation of networks that may prevent users or customers from interacting with the resources of other users or customers. Furthermore, this embodiment can allow users or customers of the system to privately store databases without relying on the public Internet 1154, which may not have the desired level of threat protection for storage.

[0124] In other embodiments, one or more LB subnets 1122 included in the control plane VCN 1116 may be configured to receive signals from the service gateway 1136. In this embodiment, the control plane VCN 1116 and the data plane VCN 1118 may be configured to be invoked by the IaaS provider's customers without invoking the public internet 1154. The IaaS provider's customers may expect this embodiment because the database(s) used by the customer can be controlled by the IaaS provider and can be stored on service lease 1119, which may be isolated from the public internet 1154.

[0125] Figure 12 This is a block diagram 1200 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 1202 (e.g., Figure 11 Service provider 1102) can communicatively couple to secure host lease 1204 (e.g., Figure 11 Secure hosting lease 1104), the secure hosting lease 1204 may include a virtual cloud network (VCN) 1206 (e.g., Figure 11 VCN 1106) and Secure Host Subnet 1208 (e.g., Figure 11 The secure host subnet 1108). VCN 1206 may include a local peering gateway (LPG) 1210 (e.g., Figure 11 The LPG 1110), VCN 1206 can be contained within the secure shell (SSH) VCN 1212 (e.g., Figure 11 LPG 1110 in SSH VCN 1112 is communicatively coupled to SSH VCN 1212. SSH VCN 1212 may include SSH subnet 1214 (e.g., Figure 11SSH subnet 1114), and SSH VCN 1212 can be accessed via VCN 1216 contained in the control plane (e.g., Figure 11 The LPG 1210 in the control plane VCN 1216 is communicatively coupled to the control plane VCN 1216. The control plane VCN 1216 may be included in the service lease 1219 (e.g., Figure 11 In the service lease 1119), and the data plane VCN 1218 (e.g., Figure 11 The data plane VCN 1118 may be included in a customer lease 1221 that may be owned or operated by the system’s users or customers.

[0126] Control plane VCN 1216 may include control plane DMZ layer 1220 (e.g., Figure 11 The control plane DMZ layer 1120), which may include one or more LB subnets 1222 (e.g., Figure 11 (One or more) LB subnets 1122), may include (one or more) application subnets 1226 (e.g., Figure 11 The control plane application layer 1224 of (one or more) application subnets 1126 (e.g., Figure 11 The control plane application layer 1124) may include one or more database (DB) subnets 1230 (e.g., similar to...). Figure 11 The control plane data layer 1228 of (one or more) DB subnets 1130 (e.g., Figure 11 The control plane data layer 1128). One or more LB subnets 1222 contained in the control plane DMZ layer 1220 can be communicatively coupled to one or more application subnets 1226 contained in the control plane application layer 1224 and an Internet gateway 1234 that can be contained in the control plane VCN 1216 (e.g., Figure 11 Internet gateway 1134), and application subnet(s) 1226 can communicatively couple to DB subnet(s) 1230 contained in control plane data layer 1228 and service gateway 1236 (e.g., Figure 11 The service gateway) and Network Address Translation (NAT) gateway 1238 (e.g., Figure 11 (NAT gateway 1138). The control plane VCN 1216 may include the service gateway 1236 and the NAT gateway 1238.

[0127] The control plane VCN 1216 may include a data plane mirror of the application layer 1240, which may include one or more application subnets 1226 (e.g., Figure 11The data plane mirror application layer 1140). One or more application subnets 1226 contained in the data plane mirror application layer 1240 may include computational instances 1244 (e.g., similar to...). Figure 11 The virtual network interface controller (VNIC) 1242 (e.g., the VNIC of 1142) of the computing instance 1144. The computing instance 1244 may facilitate the mirroring of the application subnet(s) 1226 of the application layer 1240 in the data plane and may be included in the application layer 1246 in the data plane (e.g., Figure 11 Communication between one or more application subnets 1226 in the data plane application layer 1146 via VNIC 1242 contained in the data plane mirror application layer 1240 and VNIC 1242 contained in the data plane application layer 1246.

[0128] The Internet gateway 1234, included in the control plane VCN 1216, can be communicatively coupled to the metadata management service 1252 (e.g., Figure 11 Metadata management service 1252), which can communicatively couple to public Internet 1254 (e.g., Figure 11 The public internet 1254 can communicatively couple to a NAT gateway 1238 contained in a control plane VCN 1216. The service gateway 1236 contained in the control plane VCN 1216 can communicatively couple to a cloud service 1256 (e.g., ...). Figure 11 Cloud services (1156).

[0129] In some examples, data plane VCN 1218 may be included in customer lease 1221. In this case, the IaaS provider may provide control plane VCN 1216 for each customer, and the IaaS provider may set up a unique compute instance 1244 for each customer, included in service lease 1219. Each compute instance 1244 may allow communication between control plane VCN 1216 included in service lease 1219 and data plane VCN 1218 included in customer lease 1221. Compute instance 1244 may allow resources provisioned in control plane VCN 1216 included in service lease 1219 to be deployed or otherwise used in data plane VCN 1218 included in customer lease 1221.

[0130] In other examples, an IaaS provider's customer may have a database residing in customer lease 1221. In this example, control plane VCN 1216 may include data plane mirror application layer 1240, which may include one or more application subnets 1226. Data plane mirror application layer 1240 may reside in data plane VCN 1218, but may not reside in data plane VCN 1218. In other words, data plane mirror application layer 1240 may have access to customer lease 1221, but may not reside in data plane VCN 1218 or be owned or operated by an IaaS provider's customer. Data plane mirror application layer 1240 may be configured to invoke data plane VCN 1218, but may not be configured to invoke any entity contained in control plane VCN 1216. Customers may expect to deploy or otherwise use resources provided in the control plane VCN 1216 in the data plane VCN 1218, and the data plane mirroring application layer 1240 can facilitate the customer's expected deployment or other use of resources.

[0131] In some embodiments, an IaaS provider's customer may apply filters to data plane VCN 1218. In this embodiment, the customer may determine what data plane VCN 1218 can access, and the customer may restrict access from data plane VCN 1218 to the public Internet 1254. The IaaS provider may not be able to apply filters or otherwise control data plane VCN 1218's access to any external networks or databases. Applying filters and controls to data plane VCN 1218 contained in customer lease 1221 can help isolate data plane VCN 1218 from other customers and the public Internet 1254.

[0132] In some embodiments, cloud service 1256 may be invoked by service gateway 1236 to access services that may not exist on public internet 1254, control plane VCN 1216, or data plane VCN 1218. The connection between cloud service 1256 and control plane VCN 1216 or data plane VCN 1218 may not be real-time or continuous. Cloud service 1256 may reside on different networks owned or operated by an IaaS provider. Cloud service 1256 may be configured to receive calls from service gateway 1236 and may be configured not to receive calls from public internet 1254. Some cloud services 1256 may be isolated from other cloud services 1256, and control plane VCN 1216 may be isolated from cloud services 1256 that may not be in the same region as control plane VCN 1216. For example, control plane VCN 1216 may be located in "Region 1," and cloud service "Deployment 11" may be located in both "Region 1" and "Region 2." If the service gateway 1236, contained in the control plane VCN 1216 located in region 1, makes a call to deployment 11, then that call can be transmitted to deployment 11 in region 1. In this example, the control plane VCN 1216 or deployment 11 in region 1 may not be communicatively coupled to or otherwise communicate with deployment 11 in region 2.

[0133] Figure 13 This is a block diagram 1300 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 1302 (e.g., Figure 11 Service provider 1102) can communicatively couple to secure host lease 1304 (e.g., Figure 11 Secure hosting lease 1104), the secure hosting lease 1304 may include a virtual cloud network (VCN) 1306 (e.g., Figure 11 VCN 1106) and Secure Host Subnet 1308 (e.g., Figure 11 The secure host subnet 1108). VCN 1306 may include LPG 1310 (e.g., Figure 11 The LPG 1110), VCN 1306 can be accessed via SSH VCN 1312 (e.g., LPG 1110), Figure 11 The LPG 1310 in SSH VCN 1112 is communicatively coupled to SSH VCN 1312. SSH VCN 1312 may include SSH subnet 1314 (e.g., Figure 11 SSH subnet 1114), and SSH VCN 1312 can be accessed via VCN 1316 contained in the control plane (e.g., Figure 11The LPG 1310 in the control plane VCN 1116 is communicatively coupled to the control plane VCN 1316 and via the data plane VCN 1318 (e.g., Figure 11 LPG 1310 in data plane 1118 is communicatively coupled to data plane VCN 1318. Control plane VCN 1316 and data plane VCN 1318 may be contained in service lease 1319 (e.g., Figure 11 In the service rental (1119).

[0134] The control plane VCN 1316 may include one or more load balancer (LB) subnets 1322 (e.g., Figure 11 The control plane DMZ layer 1320 of (one or more) LB subnets 1122) (e.g., Figure 11 The control plane DMZ layer 1120 may include one or more application subnets 1326 (e.g., similar to...). Figure 11 The control plane application layer 1324 of (one or more) application subnets 1126 (e.g., Figure 11 The control plane application layer 1124), and the control plane data layer 1328, which may include (one or more) DB subnets 1330, for example, Figure 11 The control plane data layer 1128). One or more LB subnets 1322 contained in the control plane DMZ layer 1320 can be communicatively coupled to one or more application subnets 1326 contained in the control plane application layer 1324 and an Internet gateway 1334 that can be contained in the control plane VCN 1316 (e.g., Figure 11 Internet gateway 1134), and application subnet(s) 1326 can communicatively couple to DB subnet(s) 1330 contained in control plane data layer 1328 and service gateway 1336 (e.g., Figure 11 The service gateway) and Network Address Translation (NAT) gateway 1338 (e.g., Figure 11 (NAT gateway 1138). The control plane VCN 1316 may include the service gateway 1336 and the NAT gateway 1338.

[0135] Data plane VCN 1318 may include data plane application layer 1346 (e.g., Figure 11 Data plane application layer 1146), data plane DMZ layer 1348 (e.g., Figure 11 Data plane DMZ layer 1148), and data plane data layer 1350 (e.g., Figure 11The data plane data layer 1150. The data plane DMZ layer 1348 may include one or more trusted application subnets 1360 and one or more untrusted application subnets 1362 that can be communicatively coupled to the data plane application layer 1346, and one or more LB subnets 1322 of the Internet gateway 1334 contained in the data plane VCN 1318. The one or more trusted application subnets 1360 may be communicatively coupled to the service gateway 1336 contained in the data plane VCN 1318, the NAT gateway 1338 contained in the data plane VCN 1318, and one or more DB subnets 1330 contained in the data plane data layer 1350. The one or more untrusted application subnets 1362 may be communicatively coupled to the service gateway 1336 contained in the data plane VCN 1318 and the one or more DB subnets 1330 contained in the data plane data layer 1350. The data plane data layer 1350 may include one or more DB subnets 1330 that can be communicatively coupled to the service gateway 1336 contained in the data plane VCN 1318.

[0136] One or more untrusted application subnets 1362 may include one or more primary VNICs 1364(1)-(N) that can be communicatively coupled to tenant virtual machines (VMs) 1366(1)-(N). Each tenant VM 1366(1)-(N) may be communicatively coupled to a corresponding application subnet 1367(1)-(N) that may be contained in a corresponding container egress VCN 1368(1)-(N) that may be contained in a corresponding customer lease 1370(1)-(N). A corresponding secondary VNIC 1372(1)-(N) may facilitate communication between one or more untrusted application subnets 1362 contained in a data plane VCN 1318 and application subnets contained in container egress VCN 1368(1)-(N). Each container exit VCN 1368(1)-(N) may include a NAT gateway 1338, which may communicatively couple to the public Internet 1354 (e.g., Figure 11 The public internet (1154).

[0137] Internet gateway 1334, contained in control plane VCN 1316 and data plane VCN 1318, can be communicatively coupled to metadata management service 1352 (e.g., Figure 11The metadata management system 1152, which is communicatively coupled to the public internet 1354, is also communicatively coupled to a NAT gateway 1338 contained in a control plane VCN 1316 and a data plane VCN 1318. A service gateway 1336 contained in both the control plane VCN 1316 and the data plane VCN 1318 is communicatively coupled to a cloud service 1356.

[0138] In some embodiments, the data plane VCN 1318 may be integrated with the customer lease 1370. Such integration may be useful or desired by the IaaS provider's customer in certain situations, such as when support may be expected during code execution. The customer may provide code that could be destructive, might communicate with other customer resources, or might otherwise cause undesirable effects. In response, the IaaS provider may determine whether to run the code provided by the customer.

[0139] In some examples, an IaaS provider's customer may grant the IaaS provider temporary network access and request functionality to be attached to data plane layer application 1346. The code running this functionality may execute in VMs 1366(1)-(N) and may not be configured to run anywhere else on data plane VCN 1318. Each VM 1366(1)-(N) may be connected to a customer lease 1370. The corresponding container 1371(1)-(N) contained in VMs 1366(1)-(N) may be configured to run the code. In this case, dual isolation may exist (e.g., container 1371(1)-(N) runs the code, where container 1371(1)-(N) may be contained in at least one or more untrusted application subnets 1362 containing VMs 1366(1)-(N)), which can help prevent incorrect or otherwise unintended code from corrupting the IaaS provider's network or the networks of different customers. Containers 1371(1)-(N) may be communicatively coupled to customer lease 1370 and may be configured to transmit or receive data from customer lease 1370. Containers 1371(1)-(N) may not be configured to transmit or receive data from any other entity in the data plane VCN 1318. After the code execution is complete, the IaaS provider may terminate or otherwise dispose of containers 1371(1)-(N).

[0140] In some embodiments, one or more trusted application subnets 1360 may run code that can be owned or operated by an IaaS provider. In this embodiment, one or more trusted application subnets 1360 may be communicatively coupled to one or more database subnets 1330 and configured to perform CRUD operations in one or more database subnets 1330. One or more untrusted application subnets 1362 may be communicatively coupled to one or more database subnets 1330, but in this embodiment, one or more untrusted application subnets may be configured to perform read operations in one or more database subnets 1330. Containers 1371(1)-(N) that may be contained in each customer's VM 1366(1)-(N) and may run code from the customer may not be communicatively coupled to one or more database subnets 1330.

[0141] In other embodiments, the control plane VCN 1316 and the data plane VCN 1318 may be coupled without direct communication. In this embodiment, there may not be direct communication between the control plane VCN 1316 and the data plane VCN 1318. However, communication may occur indirectly through at least one method. The LPG 1310 may be established by the IaaS provider, which can facilitate communication between the control plane VCN 1316 and the data plane VCN 1318. In another example, the control plane VCN 1316 or the data plane VCN 1318 may invoke the cloud service 1356 via the service gateway 1336. For example, an invocation of the cloud service 1356 from the control plane VCN 1316 may include a request for a service that can communicate with the data plane VCN 1318.

[0142] Figure 14 This is a block diagram 1400 illustrating another example pattern of an IaaS architecture according to at least one embodiment. Service operator 1402 (e.g., Figure 11 Service provider 1102) can communicatively couple to secure host lease 1404 (e.g., Figure 11 Secure hosting lease 1104), the secure hosting lease 1404 may include a virtual cloud network (VCN) 1406 (e.g., Figure 11 VCN 1106) and Secure Host Subnet 1408 (e.g., Figure 11 The secure host subnet 1108). VCN 1406 may include LPG 1410 (e.g., Figure 11 The LPG 1110), VCN 1406 can be accessed via SSH VCN 1412 (e.g., LPG 1110), Figure 11The LPG 1410 in SSH VCN 1112 is communicatively coupled to SSH VCN 1412. SSH VCN 1412 may include SSH subnet 1414 (e.g., Figure 11 SSH subnet 1114), and SSH VCN 1412 can be accessed via VCN 1416 contained in the control plane (e.g., Figure 11 The LPG 1410 in the control plane VCN 1116 is communicatively coupled to the control plane VCN 1416 and via the data plane VCN 1418 (e.g., Figure 11 LPG 1410 in data plane 1118 is communicatively coupled to data plane VCN 1418. Control plane VCN 1416 and data plane VCN 1418 may be contained in service lease 1419 (e.g., Figure 11 In the service rental (1119).

[0143] The control plane VCN 1416 may include one or more LB subnets 1422 (e.g., Figure 11 The control plane DMZ layer 1420 of (one or more) LB subnets 1122) (e.g., Figure 11 The control plane DMZ layer 1120 may include (one or more) application subnets 1426 (e.g., Figure 11 The control plane application layer 1424 of (one or more) application subnets 1126 (e.g., Figure 11 The control plane application layer 1124) may include (one or more) DB subnets 1430 (e.g., Figure 13 The control plane data layer 1428 of (one or more) DB subnets 1330 (e.g., Figure 11 The control plane data layer 1128). One or more LB subnets 1422 contained in the control plane DMZ layer 1420 can be communicatively coupled to one or more application subnets 1426 contained in the control plane application layer 1424 and an Internet gateway 1434 that can be contained in the control plane VCN 1416 (e.g., Figure 11 Internet gateway 1134), and application subnet(s) 1426 can communicatively couple to DB subnet(s) 1430 contained in control plane data layer 1428 and service gateway 1436 (e.g., Figure 11 The service gateway) and Network Address Translation (NAT) gateway 1438 (e.g., Figure 11 (NAT gateway 1138). The control plane VCN 1416 may include the service gateway 1436 and the NAT gateway 1438.

[0144] Data plane VCN 1418 may include data plane application layer 1446 (e.g., Figure 11 Data plane application layer 1146), data plane DMZ layer 1448 (e.g., Figure 11 Data plane DMZ layer 1148), and data plane data layer 1450 (e.g., Figure 11 The data plane data layer 1150). The data plane DMZ layer 1448 may include one or more trusted application subnets 1460 that can be communicatively coupled to the data plane application layer 1446 (e.g., Figure 13 (one or more) trusted application subnets 1360 and (one or more) untrusted application subnets 1462 (e.g., Figure 13 The data plane VCN 1418 may include one or more untrusted application subnets 1362 and one or more LB subnets 1422. One or more trusted application subnets 1460 may be communicatively coupled to a service gateway 1436, a NAT gateway 1438, and a DB subnet 1430, all contained in the data plane VCN 1418. One or more untrusted application subnets 1462 may be communicatively coupled to a service gateway 1436 and a DB subnet 1430, both contained in the data plane VCN 1418 and the data plane data layer 1450. The data plane data layer 1450 may include one or more DB subnets 1430 that may be communicatively coupled to a service gateway 1436, contained in the data plane VCN 1418.

[0145] One or more untrusted application subnets 1462 may include a primary VNIC 1464(1)-(N) communicatively coupled to tenant virtual machines (VMs) 1466(1)-(N) residing within one or more untrusted application subnets 1462. Each tenant VM 1466(1)-(N) may run code in a corresponding container 1467(1)-(N) and is communicatively coupled to an application subnet 1426 that may be contained in a data plane application layer 1446, which may be contained in a container egress VCN 1468. A corresponding secondary VNIC 1472(1)-(N) may facilitate communication between one or more untrusted application subnets 1462 contained in a data plane VCN 1418 and the application subnets contained in a container egress VCN 1468. The container egress VCN may include a public internet 1454 (e.g., Figure 11 The public internet (1154) uses NAT gateway 1438.

[0146] Internet gateway 1434, contained in control plane VCN 1416 and data plane VCN 1418, can be communicatively coupled to metadata management service 1452 (e.g., Figure 11 The metadata management system 1452 can be communicatively coupled to the public internet 1454. The public internet 1454 can be communicatively coupled to a NAT gateway 1438 contained in a control plane VCN 1416 and a data plane VCN 1418. The service gateway 1436 contained in the control plane VCN 1416 and the data plane VCN 1418 can be communicatively coupled to a cloud service 1456.

[0147] In some examples, Figure 14 The architecture shown in block diagram 1400 can be considered as... Figure 13 This is an exception to the pattern shown in the architecture of block diagram 1300, and is likely what the IaaS provider's customers would expect in situations where the IaaS provider cannot communicate directly with the customer (e.g., in a disconnected region). Customers can access each customer's corresponding container 1467(1)-(N) contained in VMs 1466(1)-(N) in real time. Containers 1467(1)-(N) can be configured to invoke corresponding secondary VNICs 1472(1)-(N) contained in one or more application subnets 1426 of the data plane application layer 1446, which may be contained in a container egress VCN 1468. The secondary VNICs 1472(1)-(N) can transmit the calls to a NAT gateway 1438, which can then transmit the calls to the public internet 1454. In this example, containers 1467(1)-(N), which can be accessed by clients in real time, can be isolated from the control plane VCN 1416 and from other entities contained in the data plane VCN 1418. Containers 1467(1)-(N) can also be isolated from resources from other clients.

[0148] In other examples, a client can use containers 1467(1)-(N) to invoke cloud service 1456. In this example, the client can run code within containers 1467(1)-(N) requesting a service from cloud service 1456. Container 1467(1)-(N) can then forward the request to a secondary VNIC 1472(1)-(N), which can then forward the request to a NAT gateway, which can then forward the request to the public internet 1454. The public internet 1454 can then forward the request via internet gateway 1434 to one or more LB subnets 1422 contained in control plane VCN 1416. In response to determining that the request is valid, one or more LB subnets can then forward the request to one or more application subnets 1426, which can then forward the request to cloud service 1456 via service gateway 1436.

[0149] It should be recognized that the IaaS architectures 1100, 1200, 1300, and 1400 depicted in the figures may have other components besides those depicted. Furthermore, the embodiments shown in the figures are merely some examples of cloud infrastructure systems that can be incorporated into embodiments of this disclosure. In some other embodiments, the IaaS system may have more or fewer components than shown in the figures, may combine two or more components, or may have different component arrangements or configurations.

[0150] In some embodiments, the IaaS system described herein may include application suites, middleware, and database service offerings delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such an IaaS system is the Oracle Cloud Infrastructure (OCI) provided by this assignee.

[0151] Figure 15 An example computer system 1500, in which various embodiments can be implemented, is illustrated. System 1500 can be used to implement any of the computer systems described above. As shown, computer system 1500 includes a processing unit 1504 that communicates with a plurality of peripheral subsystems via a bus subsystem 1502. These peripheral subsystems may include a processing acceleration unit 1506, an I / O subsystem 1508, a storage subsystem 1518, and a communication subsystem 1524. Storage subsystem 1518 includes a tangible computer-readable storage medium 1522 and system memory 1510.

[0152] Bus subsystem 1502 provides a mechanism for allowing various components and subsystems of computer system 1500 to communicate with each other as intended. While bus subsystem 1502 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. Bus subsystem 1502 can be any of several types of bus architectures, including memory buses or memory controllers, peripheral buses, and local buses using any of the various bus architectures. For example, such architectures may include Industry Standard Architecture (ISA) buses, Microchannel Architecture (MCA) buses, Enhanced ISA (EISA) buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses, which may be implemented as Mezzanine buses manufactured according to the IEEE P1386.1 standard.

[0153] A processing unit 1504, which may be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of the computer system 1500. One or more processors may be included in the processing unit 1504. These processors may include single-core or multi-core processors. In some embodiments, the processing unit 1504 may be implemented as one or more independent processing units 1532 and / or 1534, each including a single-core or multi-core processor. In other embodiments, the processing unit 1504 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0154] In various embodiments, processing unit 1504 can execute various programs in response to program code and can maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed can reside in processor(s) 1504 and / or storage subsystem 1518. With appropriate programming, processor(s) 1504 can provide the various functions described above. Computer system 1500 may additionally include processing acceleration unit 1506, which may include digital signal processor (DSP), dedicated processor, etc.

[0155] I / O subsystem 1508 may include user interface input devices and user interface output devices. User interface input devices may include keyboards, pointing devices such as mice or trackballs, touchpads or touchscreens integrated into a display, scroll wheels, click wheels, dials, buttons, switches, keyboards, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may include, for example, motion sensing and / or gesture recognition devices, such as the Microsoft Kinect® motion sensor, which enables users to control and interact with input devices such as the Microsoft Xbox® 360 game controller via a natural user interface using gestures and voice commands. User interface input devices may also include eye gesture recognition devices, such as the Google Glass® blink detector, which detects eye activity from the user (e.g., “blinking” when taking a photo and / or making menu selections) and translates the eye gestures into input for an input device (e.g., Google Glass®). Furthermore, user interface input devices may include voice recognition sensing devices that enable users to interact with a voice recognition system (e.g., the Siri® navigator) via voice commands.

[0156] User interface input devices may also include, but are not limited to, 3D mice, joysticks or pointing sticks, game panels and drawing tablets, and audio / video devices such as speakers, digital cameras, digital camcorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye-tracking devices. Furthermore, user interface input devices may include, for example, medical imaging input devices such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and medical ultrasound equipment. User interface input devices may also include, for example, audio input devices such as MIDI keyboards and digital musical instruments.

[0157] User interface output devices may include display subsystems, indicator lights, or non-visual displays such as audio output devices. Display subsystems may be cathode ray tubes (CRTs), flat panel devices such as those using liquid crystal displays (LCDs) or plasma displays, projection devices, touchscreens, etc. Generally, the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from computer system 1500 to a user or other computer. For example, user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0158] Computer system 1500 may include storage subsystem 1518, which provides a tangible, non-transitory, computer-readable storage medium for storing software and data constructs that provide the functionality of the embodiments described in this disclosure. The software may include programs, code, instructions, scripts, etc., which, when executed by one or more cores or processors of processing unit 1504, provide the aforementioned functionality. Storage subsystem 1518 may also provide a repository for storing data used according to this disclosure.

[0159] like Figure 15 As depicted in the example, storage subsystem 1518 may include various components, including system memory 1510, computer-readable storage medium 1522, and computer-readable storage medium reader 1520. System memory 1510 may store program instructions that can be loaded and executed by processing unit 1504. System memory 1510 may also store data used during the execution of instructions and / or data generated during the execution of program instructions. Various types of programs may be loaded into system memory 1510, including but not limited to client applications, web browsers, middleware applications, relational database management systems (RDBMS), virtual machines, containers, etc.

[0160] System memory 1510 may also store operating system 1516. Examples of operating system 1516 may include various versions of Microsoft Windows®, Apple Macintosh® and / or Linux operating systems, various commercially available UNIX® or UNIX-like operating systems (including, but not limited to, various GNU / Linux operating systems, Google Chrome® OS, etc.) and / or mobile operating systems such as iOS, Windows® Phone, Android® OS, BlackBerry® OS, and Palm® OS. In some implementations of computer system 1500 that execute one or more virtual machines, the virtual machine, along with the guest operating system (GOS), may be loaded into system memory 1510 and executed by one or more processors or cores of processing unit 1504.

[0161] System memory 1510 can be configured differently depending on the type of computer system 1500. For example, system memory 1510 can be volatile memory (such as random access memory (RAM)) and / or non-volatile memory (such as read-only memory (ROM), flash memory, etc.). Different types of RAM configurations can be provided, including static random access memory (SRAM), dynamic random access memory (DRAM), etc. In some implementations, system memory 1510 may include a basic input / output system (BIOS), which contains basic routines such as those that facilitate the transfer of information between components within computer system 1500 during startup.

[0162] Computer-readable storage medium 1522 may represent remote, local, fixed and / or removable storage devices and storage media for temporarily and / or more permanently containing and storing computer-readable information for use by computer system 1500, including instructions executable by processing unit 1504 of computer system 1500.

[0163] Computer-readable storage medium 1522 may include any suitable medium known or used in the art, including storage media and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing and / or transmitting information. This may include tangible computer-readable storage media such as RAM, ROM, electrically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or other tangible computer-readable media.

[0164] As an example, computer-readable storage medium 1522 may include a hard disk drive that reads or writes to a non-removable non-volatile magnetic medium, a disk drive that reads or writes to a removable non-volatile magnetic disk, and an optical disc drive that reads or writes to a removable non-volatile optical disc (such as CD ROM, DVD, and Blu-ray® discs or other optical media). Computer-readable storage medium 1522 may include, but is not limited to, Zip® drives, flash memory cards, Universal Serial Bus (USB) flash drives, Secure Digital (SD) cards, DVD discs, digital audio tapes, and so on. Computer-readable storage medium 1522 may also include solid-state drives (SSDs) based on non-volatile memory (such as flash memory-based SSDs, enterprise flash drives, solid-state ROMs, etc.), volatile memory-based SSDs (such as solid-state RAM, dynamic RAM, static RAM), DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs, and hybrid SSDs using a combination of DRAM-based and flash memory-based SSDs. Disk drives and their associated computer-readable media can provide non-volatile storage for computer-readable instructions, data structures, program services and other data for computer system 1500.

[0165] Machine-readable instructions executable by one or more processors or cores of processing unit 1504 may be stored on a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium may include physically tangible memory or storage devices, including volatile memory storage devices and / or non-volatile memory devices. Examples of non-transitory computer-readable storage media include magnetic storage media (e.g., disks or tapes), optical storage media (e.g., DVDs, CDs), various types of RAM, ROM, or flash memory, hard disk drives, floppy disk drives, removable memory drives (e.g., USB drives), or other types of storage devices.

[0166] The communication subsystem 1524 provides an interface to other computer systems and networks. The communication subsystem 1524 serves as an interface for receiving data from other systems and transmitting data from computer system 1500 to other systems. For example, the communication subsystem 1524 enables computer system 1500 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 1524 may include a radio frequency (RF) transceiver component (e.g., using cellular telephone technology, advanced data network technologies such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 802.11 series standards), or other mobile communication technologies, or any combination thereof), a global positioning system (GPS) receiver component, and / or other components for accessing wireless voice and / or data networks. In some embodiments, the communication subsystem 1524 may provide a wired network connection (e.g., Ethernet) as an addition to or alternative to the wireless interface.

[0167] In some embodiments, the communication subsystem 1524 may also represent one or more users who can use the computer system 1500 to receive input communications in the form of structured and / or unstructured data feeds 1526, event streams 1528, event updates 1530, etc.

[0168] For example, the communication subsystem 1524 can be configured to receive data feeds 1526 in real time from users of social networks and / or other communication services, such as Twitter® feeds, Facebook® updates, web feeds such as Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0169] Furthermore, the communication subsystem 1524 can also be configured to receive data in the form of a continuous data stream, which may include event streams 1528 and / or event updates 1530 that are essentially continuous or unbounded real-time events without a clearly defined termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial quotation machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, vehicle traffic monitoring, etc.

[0170] The communication subsystem 1524 can also be configured to output structured and / or unstructured data feeds 1526, event streams 1528, event updates 1530, etc. to one or more databases, which can communicate with one or more streaming data source computers coupled to the computer system 1500.

[0171] The computer system 1500 can be one of a variety of types, including handheld portable devices (e.g., iPhone® cellular phones, iPad® computing tablets, PDAs), wearable devices (e.g., Google Glass® head-mounted displays), PCs, workstations, mainframes, information stations, server racks, or any other data processing systems.

[0172] Due to the constantly evolving nature of computers and networks, the description of the computer system 1500 depicted in the figures is intended only as a specific example. Many other configurations with more or fewer components than the system depicted in the figures are possible. For example, custom hardware may also be used and / or specific elements may be implemented in hardware, firmware, software (including applets), or a combination thereof. Additionally, connectivity with other computing devices, such as network input / output devices, may be employed. Based on the disclosure and teachings provided herein, those skilled in the art will recognize other ways and / or methods for implementing the various embodiments.

[0173] While specific embodiments have been described, various modifications, alterations, alternative constructions, and equivalents are also included within the scope of this disclosure. The embodiments are not limited to operation within certain specific data processing environments, but can be freely operated within multiple data processing environments. Furthermore, although the embodiments have been described using a specific series of transactions and steps, those skilled in the art will understand that the scope of this disclosure is not limited to the described series of transactions and steps. Various features and aspects of the above embodiments can be used individually or in combination.

[0174] Furthermore, while embodiments have been described using specific combinations of hardware and software, it should be recognized that other combinations of hardware and software are also within the scope of this disclosure. Embodiments may be implemented using only hardware, or only software, or a combination thereof. The various processes described herein can be implemented in any combination on the same processor or on different processors. Thus, where a component or service is described as being configured to perform certain operations, such configuration can be accomplished, for example, by designing electronic circuits, by programming programmable electronic circuits (such as microprocessors), or by any combination thereof. Processes may communicate using a variety of technologies, including but not limited to conventional technologies for inter-process communication, and different pairs of processes may use different technologies, or the same pair of processes may use different technologies at different times.

[0175] Therefore, the specification and drawings are to be considered illustrative rather than restrictive. However, additions, omissions, deletions, and other modifications and changes may be made therein without departing from the broader spirit and scope set forth in the claims. Thus, while specific disclosed embodiments have been described, they are not intended to be limiting. Various modifications and equivalents are within the scope of the following claims.

[0176] In the context of describing the disclosed embodiments (particularly in the context of the following claims), the terms “a,” “an,” and “the,” and similar designations, should be interpreted as encompassing both the singular and plural, unless otherwise indicated herein or clearly contradicted by the context. Unless otherwise stated, the terms “comprising,” “having,” “including,” and “containing” should be interpreted as open-ended terms (i.e., meaning “including but not limited to”). The term “connected” should be interpreted as partially or wholly contained, attached to, or joined together, even if something exists in between. Unless otherwise indicated herein, the description of value ranges herein is intended only as a concise way of referring to each individual value falling within that range, and each individual value is incorporated into the specification as if it were described separately herein. Unless otherwise indicated herein or clearly contradicted by the context, all methods described herein can be performed in any suitable order. Unless otherwise stated, the use of any and all examples or exemplary language (e.g., “such as”) provided herein is intended only to better illustrate the embodiments and does not constitute a limitation on the scope of this disclosure. No language in the specification should be construed as indicating that any unclaimed element is essential to the practice of this disclosure.

[0177] Unless otherwise expressly stated, disjunctive language (such as the phrase "at least one of X, Y, or Z") is intended to be understood in context as a general indication that an item, term, etc., can be any one of X, Y, or Z or any combination thereof (e.g., X, Y, and / or Z). Therefore, such disjunctive language is generally not intended, nor should it imply, that certain embodiments require the presence of at least one of X, at least one of Y, or at least one of Z, each individually.

[0178] This document describes preferred embodiments of the present disclosure, including the best modes known for carrying out the present disclosure. Variations of these preferred embodiments will become apparent to those skilled in the art upon reading the foregoing description. Those skilled in the art should be able to suitably employ such variations and may practice the present disclosure in ways other than those specifically described herein. Thus, the present disclosure includes all modifications and equivalents to the subject matter recited in the appended claims, where permitted by applicable law. Furthermore, unless otherwise indicated herein, the present disclosure includes any combination of the foregoing elements in all possible variations.

[0179] All references cited in this article (including publications, patent applications and patents) are incorporated into this article by reference to the same extent as if each reference were individually and specifically incorporated by reference and elaborated in the entire article.

[0180] In the foregoing specification, various aspects of this disclosure have been described with reference to specific embodiments thereof; however, those skilled in the art will recognize that this disclosure is not limited thereto. The various features and aspects of the foregoing disclosure may be used individually or in combination. Furthermore, embodiments may be used in any number of environments and applications other than those described herein without departing from the broader spirit and scope of this specification. Therefore, this specification and the accompanying drawings should be considered illustrative rather than restrictive.

Claims

1. A computer-implemented method, comprising: By obtaining at least a first public encryption key from a first cloud computing edge device, the edge device cloud service associates the first cloud computing edge device with a cluster of cloud computing edge devices, the cluster being characterized by a master encryption key and a public encryption key obtained from the cloud computing edge device; The first cloud computing edge device, which carries the master encryption key, is supplied by the edge device cloud service. The master encryption key is stored in the first key repository of the first cloud computing edge device. By obtaining at least the second public encryption key of the second cloud computing edge device, the edge device cloud service associates the second cloud computing edge device with the cluster; The second cloud computing edge device, which carries a master encryption key and a first public encryption key, is supplied by the edge device cloud service. The master encryption key and the first public encryption key are stored in the second key repository of the second cloud computing edge device. The first cloud computing edge device receives encrypted message data from the second cloud computing edge device. The encrypted message data includes a second public encryption key and is generated by the second cloud computing edge device using a master encryption key. The encrypted message data is decrypted by the first cloud computing edge device using the master encryption key stored in the first key repository; as well as The first key repository is updated by the first cloud computing edge device using the second public encryption key.

2. The computer-implemented method as described in claim 1, further comprising: By obtaining at least the third public encryption key of the third cloud computing edge device, the edge device cloud service associates the third cloud computing edge device with the cluster; as well as The edge device cloud service provides a third cloud computing edge device with a master encryption key, a first public encryption key, and a second public encryption key.

3. The computer-implemented method as described in claim 1 or claim 2, further comprising: A request for a session token is received by a first cloud computing edge device from a second cloud computing edge device, the request including a signature generated using a second private encryption key corresponding to a second public encryption key; The signature is verified by a first cloud computing edge device using a second public encryption key stored in a first key repository; as well as Based at least in part on the successful verification of the signature, the first cloud computing edge device sends the session token to the second cloud computing edge device.

4. The computer-implemented method of claim 3, wherein the encrypted message data further includes a network address corresponding to the second cloud computing edge device, and wherein the computer-implemented method further includes: The request is verified by the first cloud computing edge device by comparing the network address received in the encrypted message data with the network address of the second cloud computing edge device.

5. The computer-implemented method as described in claim 3 or claim 4, further comprising: The first cloud computing edge device sends an application programming interface (API) request to the second cloud computing edge device, the API request including a session token; as well as The first cloud computing edge device receives an indication from the second cloud computing edge device that the API request has been successfully completed, the indication being generated at least in part based on the successful verification of the session token.

6. The computer-implemented method as described in any of the preceding claims, further comprising: The first cloud computing edge device receives additional encrypted message data from the second cloud computing edge device. The additional encrypted message data includes an updated public encryption key and is encrypted using a master encryption key. The updated public encryption key is generated by the second cloud computing edge device. as well as The first key repository is updated by the first cloud computing edge device with the updated public encryption key.

7. The computer-implemented method as described in any of the preceding claims, further comprising: The edge device cloud service provides the first cloud computing edge device with a cluster policy that specifies multiple permissible actions of the cluster of cloud computing edge devices; as well as The first cloud computing edge device receives a request from the second cloud computing edge device that can be used to perform an action; as well as The action is performed in response to the request by a first cloud computing edge device and at least in part based on determining that the action corresponding to the request is one of the permissible actions.

8. A distributed computing system, comprising: One or more processors; as well as One or more memories storing computer-executable instructions, which, when executed by the one or more processors, cause the distributed computing system to perform the following operations: By obtaining at least a first public encryption key from a first cloud computing edge device, the edge device cloud service associates the first cloud computing edge device with a cluster of cloud computing edge devices, the cluster being characterized by a master encryption key and a public encryption key obtained from the cloud computing edge device; The first cloud computing edge device, which carries the master encryption key, is supplied by the edge device cloud service. The master encryption key is stored in the first key repository of the first cloud computing edge device. By obtaining at least the second public encryption key of the second cloud computing edge device, the edge device cloud service associates the second cloud computing edge device with the cluster; The second cloud computing edge device, which carries a master encryption key and a first public encryption key, is supplied by the edge device cloud service. The master encryption key and the first public encryption key are stored in the second key repository of the second cloud computing edge device. The first cloud computing edge device receives encrypted message data from the second cloud computing edge device. The encrypted message data includes a second public encryption key and is generated by the second cloud computing edge device using a master encryption key. The encrypted message data is decrypted by the first cloud computing edge device using the master encryption key stored in the first key repository; as well as The first key repository is updated by the first cloud computing edge device using the second public encryption key.

9. The distributed computing system of claim 8, wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the distributed computing system to further perform the following operations: By obtaining at least the third public encryption key of the third cloud computing edge device, the edge device cloud service associates the third cloud computing edge device with the cluster; and The edge device cloud service provides a third cloud computing edge device with a master encryption key, a first public encryption key, and a second public encryption key.

10. The distributed computing system of claim 8 or claim 9, wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the distributed computing system to further perform the following operations: A request for a session token is received by a first cloud computing edge device from a second cloud computing edge device, the request including a signature generated using a second private encryption key corresponding to a second public encryption key; The signature is verified by a first cloud computing edge device using a second public encryption key stored in a first key repository; as well as Based at least in part on the successful verification of the signature, the first cloud computing edge device sends the session token to the second cloud computing edge device.

11. The distributed computing system of claim 10, wherein the encrypted message data further includes a network address corresponding to a second cloud computing edge device, and wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the distributed computing system to further verify the request by at least comparing the network address received in the encrypted message data with the network address of the second cloud computing edge device.

12. The distributed computing system of claim 10 or claim 11, wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the distributed computing system to further perform the following operations: A first cloud computing edge device sends an application programming interface (API) request to a second cloud computing edge device, the API request including a session token; and The first cloud computing edge device receives an indication from the second cloud computing edge device that the API request has been successfully completed, the indication being generated at least in part based on the successful verification of the session token.

13. The distributed computing system of any one of claims 8 to 12, wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the distributed computing system to further perform the following operations: The first cloud computing edge device receives additional encrypted message data from the second cloud computing edge device. This additional encrypted message data includes an updated public encryption key and is encrypted using a master encryption key. The updated public encryption key is generated by the second cloud computing edge device. The first key repository is updated by the first cloud computing edge device with the updated public encryption key.

14. The distributed computing system of any one of claims 8 to 13, wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the distributed computing system to further perform the following operations: The edge device cloud service provides the first cloud computing edge device with a cluster policy that specifies multiple permissible actions of the cluster of cloud computing edge devices; as well as The first cloud computing edge device receives a request from the second cloud computing edge device that can be used to perform an action; as well as The action is performed in response to the request by a first cloud computing edge device and at least in part based on determining that the action corresponding to the request is one of the permissible actions.

15. A non-transitory computer-readable medium comprising executable instructions, which, when executed by one or more processors of a distributed computing system, cause the distributed computing system to perform the following operations: By obtaining at least a first public encryption key from a first cloud computing edge device, the edge device cloud service associates the first cloud computing edge device with a cluster of cloud computing edge devices, the cluster being characterized by a master encryption key and a public encryption key obtained from the cloud computing edge device; The first cloud computing edge device, which carries the master encryption key, is supplied by the edge device cloud service. The master encryption key is stored in the first key repository of the first cloud computing edge device. By obtaining at least the second public encryption key of the second cloud computing edge device, the edge device cloud service associates the second cloud computing edge device with the cluster; The second cloud computing edge device, which carries a master encryption key and a first public encryption key, is supplied by the edge device cloud service. The master encryption key and the first public encryption key are stored in the second key repository of the second cloud computing edge device. The first cloud computing edge device receives encrypted message data from the second cloud computing edge device. The encrypted message data includes a second public encryption key and is generated by the second cloud computing edge device using a master encryption key. The encrypted message data is decrypted by the first cloud computing edge device using the master encryption key stored in the first key repository; as well as The first key repository is updated by the first cloud computing edge device using the second public encryption key.

16. The non-transitory computer-readable medium of claim 15, further comprising additional instructions that, when executed by the one or more processors, cause the distributed computing system to perform the following operations: By obtaining at least the third public encryption key of the third cloud computing edge device, the edge device cloud service associates the third cloud computing edge device with the cluster; and The edge device cloud service provides a third cloud computing edge device with a master encryption key, a first public encryption key, and a second public encryption key.

17. The non-transitory computer-readable medium of claim 15 or claim 16, further comprising additional instructions that, when executed by the one or more processors, cause the distributed computing system to perform the following operations: A request for a session token is received by a first cloud computing edge device from a second cloud computing edge device, the request including a signature generated using a second private encryption key corresponding to a second public encryption key; The signature is verified by a first cloud computing edge device using a second public encryption key stored in a first key repository; as well as Based at least in part on the successful verification of the signature, the first cloud computing edge device sends the session token to the second cloud computing edge device.

18. The non-transitory computer-readable medium of claim 17, wherein the encrypted message data further includes a network address corresponding to a second cloud computing edge device, and the non-transitory computer-readable medium includes additional instructions that, when executed by the one or more processors, cause the distributed computing system to further verify the request by at least comparing the network address received in the encrypted message data with the network address of the second cloud computing edge device.

19. The non-transitory computer-readable medium of claim 17 or claim 18, further comprising additional instructions that, when executed by the one or more processors, cause the distributed computing system to perform the following operations: A first cloud computing edge device sends an application programming interface (API) request to a second cloud computing edge device, the API request including a session token; and The first cloud computing edge device receives an indication from the second cloud computing edge device that the API request has been successfully completed, the indication being generated at least in part based on the successful verification of the session token.

20. The non-transitory computer-readable medium of any one of claims 15 to 19, comprising additional instructions that, when executed by the one or more processors, cause the distributed computing system to further perform the following operations: The first cloud computing edge device receives additional encrypted message data from the second cloud computing edge device. This additional encrypted message data includes an updated public encryption key and is encrypted using a master encryption key. The updated public encryption key is generated by the second cloud computing edge device. The first key repository is updated by the first cloud computing edge device with the updated public encryption key.

Images