Methods, apparatus, and products for workload allocation in edge environments

Abstract

To provide methods, apparatus, systems and articles of manufacture for a workload placement in an edge environment.SOLUTION: An apparatus for a workload placement in an edge environment includes an orchestrator to receive a request to execute a workload from an edge platform within an edge environment, and a capability controller to analyze the request to determine operating parameters for the workload from the edge platform, and analyze candidate edge tier and edge platform placements based on the operating parameters. The orchestrator determines a candidate edge tier and edge platform placement for the workload based on a candidate edge tier and edge platform placement that satisfies the operating parameters.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates generally to edge environments, and more particularly, to methods, apparatus, and products for workload placement in edge environments.

Background Art

[0002] An edge environment (e.g., edge, network edge, fog computing, multi-access edge computing (MEC), or Internet of Things (IoT) network) enables workload execution (e.g., execution of one or more computing tasks, execution of a machine learning model using input data, etc.) closer to, or nearer to, an endpoint device that requires the execution of that workload. The edge environment may include infrastructure (e.g., network infrastructure) such as edge services that are connected via a network such as the Internet to cloud infrastructure, endpoint devices, or additional edge infrastructure. Edge services may be closer to endpoint devices than cloud infrastructure such as a centralized server.

Brief Description of the Drawings

[0003] [Figure 1] An exemplary edge computing system for providing edge services and applications in accordance with the teachings of the present disclosure is shown.

[0004] [Figure 2] An exemplary implementation of an edge platform for processing requests received from a client computing node in accordance with the teachings of the present disclosure is shown.

[0005] [Figure 3]This flowchart shows exemplary machine-readable instructions that may be executed to implement the exemplary orchestrator in Figures 2 and / or 3 to determine candidate edge layer and edge platform placements for a workload.

[0006] [Figure 4] This flowchart shows exemplary machine-readable instructions that may be executed to implement the exemplary orchestrator in Figures 2 and / or 3 to analyze requests in order to determine the operational parameters of the workload to be implemented on the edge platform.

[0007] [Figure 5] This flowchart shows exemplary machine-readable instructions that may be executed to implement the exemplary orchestrator in Figures 2 and / or 3 to analyze candidate edge layer and edge platform placements based on operating parameters.

[0008] [Figure 6] This flowchart shows exemplary machine-readable instructions that may be executed to implement the exemplary orchestrator in Figures 2 and / or 3 to determine candidate edge layer and edge platform placements for a workload.

[0009] [Figure 7] This is a block diagram of an exemplary processing platform configured to execute the instructions in Figure 3 in order to implement the orchestrator in Figure 2.

[0010] [Figure 8]This is a block diagram of an exemplary software distribution platform for distributing software (for example, software corresponding to the exemplary computer-readable instructions in Figures 3 to 6) to client devices, such as consumers (for example, for licensing, sales, and / or use), retailers (for example, for sales, resale, licensing, and / or sublicensing), and / or original equipment manufacturers (OEMs) (for example, for inclusion in products delivered to retailers and / or direct purchase customers).

[0011] [Figure 9] This disclosure provides an overview of edge cloud configurations for edge computing based on the teachings presented here.

[0012] [Figure 10] The teachings in this disclosure illustrate the operational layers between endpoints, edge cloud, and cloud computing environments.

[0013] [Figure 11] A block diagram of an exemplary environment for networking and services in an edge computing system, as taught in this disclosure, is shown.

[0014] [Figure 12] The teachings of this disclosure demonstrate the deployment of a virtual edge configuration in an edge computing system operating between multiple edge nodes and multiple tenants.

[0015] [Figure 13] The teachings of this disclosure illustrate various computing configurations for deploying containers in edge computing systems.

[0016] [Figure 14]Illustrative computing - communication use cases related to mobile access to an application in an exemplary edge - computing system according to the teachings of the present disclosure are shown.

[0017] [Figure 15A] FIG. 6 is a block diagram of an exemplary implementation of an exemplary computing node that can be deployed in one of the edge - computing systems shown in FIGS. 9 - 12 and / or FIG. 14 according to the teachings of the present disclosure.

[0018] [Figure 15B] FIG. 7 is another block diagram of an exemplary implementation of an exemplary computing node that can be deployed in one of the edge - computing systems shown in FIGS. 9 - 12 and / or FIG. 14 according to the teachings of the present disclosure.

[0019] The drawings are not to scale. Generally, the same reference numbers are used throughout the drawings and the accompanying written description to refer to the same or similar parts. References to connections (e.g., attached, coupled, connected, linked) should be construed broadly and may, unless otherwise indicated, include intermediate members between assemblies of elements and relative movement between elements. Thus, references to connections do not necessarily presume that two elements are directly connected and in a fixed relationship with each other.

[0020] Unless otherwise indicated, descriptors such as "first", "second", "third", etc. are used in this specification without any sense of priority, physical order, arrangement within a list, and / or ordering in any way, or otherwise indicated, and are merely used as labels and / or arbitrary names to distinguish elements for ease of understanding of the disclosed examples.

BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Edge computing, at a general level, refers to the shift of computing and storage resources closer to endpoint devices (e.g., consumer computing devices, user equipment, etc.) and is adopted to optimize total cost of ownership, reduce application latency, improve service functionality, and enhance compliance with data privacy or security requirements. In some scenarios, edge computing provides a cloud-like distributed service that offers orchestration and management for applications across many types of storage and computing resources. As a result, some implementations of edge computing are called "edge cloud" or "fog" because powerful computing resources that were previously only available in large remote data centers are moved closer to endpoints and made available for consumer use at the "edge" of the network.

[0022] Use cases for edge computing in mobile network settings have been developed to integrate with the multi-access edge computing (MEC) approach, also known as "mobile edge computing." The MEC approach is designed to enable application developers and content providers to access computing capabilities and information technology (IT) service environments at the network edge in dynamic mobile network settings. Limited standards have been developed by the European Telecommunications Standards Institute (ETSI) Industry Specifications Group (ISG) in an attempt to define a common interface for the operation of MEC systems, platforms, hosts, services, and applications.

[0023] Edge computing, MEC, and related technologies aim to deliver lower latency, improved responsiveness, and greater computing power than traditional cloud network services or wide-area network connectivity. However, integrating mobility and dynamically launched services into several mobile and device processing use cases has introduced limitations and concerns regarding orchestration, functional coordination, and resource management, particularly in complex mobile scenarios involving multiple participants (e.g., devices, hosts, tenants, service providers, operators, etc.).

[0024] Similarly, Internet of Things (IoT) networks and devices are designed to provide a distributed computing configuration from diverse endpoints. IoT devices may be physical or virtual objects that can communicate over the network and may include sensors, actuators, and other input / output components that can be used to collect data or perform actions in a real-world environment. For example, IoT devices may include low-power endpoint devices that are embedded in or attached to everyday objects such as buildings, vehicles, and packages, which can provide an additional level of artificial sensory perception of those objects. In recent years, IoT devices have become more common, and applications that use these devices have become widespread.

[0025] In some examples, the edge environment can include the enterprise edge, and communication with and / or within the enterprise edge can be facilitated via wireless and / or wired connections. The deployment of various edge, fog, MEC, and IoT networks, devices, and services has introduced many advanced use cases and scenarios occurring at and near the edge of the network. However, these advanced use cases have also brought several corresponding technical challenges, among many others, relating to security, processing and network resources, and service availability and efficiency. One such challenge concerns edge, fog, MEC, and IoT networks, devices, and services that perform workloads on behalf of endpoint devices.

[0026] The techniques and configurations of this application may be used in relation to many aspects of current networked systems, but are provided with reference to edge cloud, IoT, multi-access edge computing (MEC), and other distributed computing deployments. The following systems and techniques may be implemented in or augment a variety of distributed, virtualized, or managed edge computing systems. These include environments in which network services are implemented or managed using multi-access edge computing (MEC), fourth-generation (4G) or fifth-generation (5G) wireless network configurations, or in wired network configurations involving fiber, copper, and other connectivity. Furthermore, the aspects of processing by each computing component may involve computing elements geographically close to user equipment or other endpoint locations, such as smartphones, vehicle communication components, and IoT devices. In addition, the techniques of this disclosure may relate to other edge / MEC / IoT network communication standards and configurations, as well as other intermediate processing entities and architectures.

[0027] Edge computing is an evolving paradigm in which computing is performed at or near the “edge” of the network, typically through the use of computing platforms implemented in base stations, gateways, network routers, or other devices much closer to endpoint devices that generate and consume data. For example, an edge gateway server may have a pool of memory and storage resources to perform real-time computations for low-latency use cases for connected client devices (e.g., autonomous driving or video surveillance). Alternatively, as an example, a base station may be augmented with computing and acceleration resources to directly handle service workloads for connected user equipment without further data transmission over the backhaul network. Or, as another example, central office network management hardware may be replaced with computing hardware that performs virtualized network functions and provides computing resources for performing service and consumer functions for connected devices.

[0028] The edge environment includes the network and / or a portion of the network located between the cloud environment and the endpoint environment. The edge environment enables the computation of workloads at the edge of the network. For example, an endpoint device may request that a nearby base station compute its workload rather than a central server in the cloud environment. The edge environment includes an edge platform that includes a pool of memory, storage resources, and / or processing resources. The edge platform performs computations, such as workload execution, on behalf of other edge platforms and / or edge nodes. The edge environment facilitates connectivity between producers (e.g., workload executors, edge platforms) and consumers (e.g., other edge platforms, endpoint devices).

[0029] Because edge platforms may be closer to endpoint devices than centralized servers in a cloud environment, they can enable workload calculations with lower latency (e.g., response time) than cloud environments, while also experiencing less communication delay and network utilization for transmitting data, computation requests, and results. Edge platforms can also enable localized execution of workloads based on geographical location or network topography. For example, an endpoint device might require that a workload be executed in a first geographical area, while a centralized server might be located in a second geographical area. The endpoint device might then require that the workload be executed by an edge platform located in the first geographical area to comply with enterprise or regulatory constraints.

[0030] Examples of workloads performed in edge environments include autonomous driving calculations, video surveillance monitoring, machine learning model execution, and real-time data analysis. Further examples of workloads include media stream delivery and / or encoding, ad impression rate measurement, object detection in media streams, speech analysis, asset and / or inventory management, and augmented reality processing.

[0031] Edge platforms enable both the execution of workloads and the return of the results of those workloads to endpoint devices with shorter response times than servers in a cloud environment. For example, if the edge platform is located closer to the endpoint devices on the network than the cloud servers, the edge service may respond to workload execution requests from the endpoint devices faster than the cloud servers. Endpoint devices may request the execution of time-sensitive workloads from the edge service rather than the cloud server.

[0032] Furthermore, edge platforms enable the distribution and decentralization of workload execution. For example, an endpoint device may request a first workload execution and a second workload execution. In some examples, a cloud server can respond to both workload execution requests. However, in an edge environment, the first edge platform may execute the first workload execution request, and the second edge platform may execute the second workload execution request.

[0033] To meet the low latency and / or high bandwidth requirements of endpoint devices, orchestration in the edge cloud is performed based on timely information about the utilization of many resources (e.g., hardware resources, software resources, virtual hardware and / or software resources) and the efficiency with which those resources can meet the requirements imposed on the endpoint devices. Such timely information is generally referred to as telemetry (e.g., telemetry data, telemetry information).

[0034] Telemetry can be generated from multiple sources, including each hardware component or part thereof, virtual machines (VMs), operating systems (OSs), applications, and orchestrators. Telemetry can be used by orchestrators, schedulers, etc., to determine, based on historical and / or current (instantaneous or near-instantaneous) telemetry, the number, quantity, and / or type of computational tasks scheduled to run on which resources or parts thereof, and the estimated time to completion of such computational tasks. For example, the cores of a multicore central processing unit (CPU) can generate thousands of different pieces of information every second or less, using a performance monitoring unit (PMU) that samples the core and / or more generally the multicore CPU. Periodically aggregating and processing all such telemetry on a given edge platform, edge node, etc., can be a difficult and cumbersome process. Prioritizing and extracting prominent features of interest from the telemetry to identify current or future problems, stressors, etc., related to the resources is difficult. Furthermore, identifying alternative resources to offload workloads from heavily used resources is a complex task.

[0035] Some edge environments desire to obtain telemetry data associated with resources performing diverse functions or services, such as data processing or video analytics capabilities (e.g., machine vision, image processing for autonomous vehicles, facial recognition detection, visual object detection, etc.). However, many high-throughput workloads, including one or more video analytics functions, can be executed in less than a millisecond (or other relatively short durations). Such edge environments lack distributed monitoring software or hardware solutions, or combinations thereof, that can monitor such highly-granular, stateless functions running on a platform (e.g., resource platform, hardware platform, software platform, virtualization platform, etc.).

[0036] Many edge environments include a variety of components for resource management and orchestration.

[0037] Given the power and / or thermal constraints at edge platforms (e.g., base stations), which differ from more traditional centralized cloud environments (e.g., central offices), dynamic, intelligent, and per-tenant power management policies at edge platforms can reduce and / or recover capital and / or operational expenses associated with edge architectures. For example, by monetizing all the functionality invested in an edge service provider's edge architecture, the edge provider can recover the capital and / or operational expenses associated with the functionality of the edge architecture. Some edge architectures can be powered by solar and wind energy. If computing resources and / or thermal conditions at an edge platform are powered by variable renewable energy (e.g., solar, wind, hydro, etc.) and / or with limited-capacity battery backup, the reliability of the edge platform may be reduced if precise power management for services cannot be provided. Furthermore, while some edge architectures can have stable power (e.g., connected to a grid), balancing thermal conditions can be challenging for such edge architectures.

[0038] While original equipment manufacturers (OEMs) and silicon vendors consider the power requirements and / or power supplies of distributed computing environments, many assume continuously powered environments, such as data centers, with uninterruptible power supplies or generators available for support during outages. Many OEM and silicon vendor designs for edge platforms lack support for optimal and flexible operation under dynamic power and / or thermal envelopes. Furthermore, edge platforms can have different power performance implications when operating in traditional computing environments rather than edge environments.

[0039] Another challenge in edge environments is limited supply, not only in terms of power but also in terms of the elasticity of the edge platform. When extending traditional data center-like cloud practices to hosting applications at various edge locations, factors to consider include: the number of resources allocated to each workload (e.g., service, application) on the edge platform; where and / or how to utilize accelerators to achieve good performance per watt, and which services to migrate between edge platforms to prevent spikes in power consumption; and how to balance power demand across service level agreements related to various tenant workloads (e.g., based on policy).

[0040] The non-uniform, unpredictable demands that can arise in edge environments, along with the inelastic supply of power and / or other resources in the edge environment, cause not only user / tenant services / applications to consume power and / or other resources, but also system software stacks and edge platform management components to consume power and hardware. For example, in some edge platforms, the software stack and edge platform management components may occupy 30% of the overall edge platform footprint.

[0041] Examples disclosed herein include methods, apparatus, and products for workload placement in an edge environment for controlling the processing of requests for workload execution on an edge platform. Examples disclosed herein include receiving a request to execute a workload from an edge platform in an edge environment. Examples disclosed herein include analyzing the request to determine operational parameters for the workload from the edge platform. In some examples, determining the operational parameters includes deriving a subset of factors necessary for the operational parameters. In some examples, the subset of factors includes security levels, accuracy levels, dependencies of the edge platform to other microservices, location of dependencies in the edge environment, usage information of the edge platform, or service level agreement (SLA) attributes of the edge platform. In some examples, determining the operational parameters includes estimating resource requirements for the edge platform, the resource requirements including power measurements, thermal measurements, latency measurements, and / or bandwidth measurements for the workload based on the subset of factors. In some examples, the resource requirements are updated based on updated operational parameters.

[0042] Examples disclosed herein include analyzing candidate edge layer and edge platform placements for a workload based on operational parameters. In some examples, analyzing candidate edge layer and edge platform placements includes determining the resource availability of each candidate edge layer and edge platform placement. In some examples, resource availability corresponds to power availability, thermal availability, latency availability, and / or bandwidth availability at each candidate edge layer and edge platform placement, based on telemetry data from each of them. Examples disclosed herein include determining candidate edge layer and edge platform placements for a workload based on candidate edge layer and edge platform placements that satisfy operational parameters. In some examples, determining candidate edge layers and edge platform placements for an edge platform involves: determining the operational parameter ratio for each of the candidate edge layers and edge platform placements, where the operational parameter ratio corresponds to the percentage of resource availability for the edge layers and edge platform placements that meet the resource requirements of the edge platform; selecting the edge layer and edge platform placement with the optimal operational parameter ratio as the candidate edge layer and edge platform placement; and executing the workload on the candidate edge layers and edge platform placement. In some examples, the resource requirements and cost function weights are adjusted in response to the operational parameter ratio for each of the candidate edge layers and edge platform placements not meeting a threshold. As used herein, “cost function weight” or “cost function” refers to a penalty that increases with the magnitude of the error (e.g., the number of iterations) to ensure accuracy and security during the placement and execution of the workload.

[0043] In some examples, cost function weights are used to progressively bias the selection of candidate edge layer and edge platform placements to be more conservative. More conservative means applying higher cost function weights, which corresponds to allocating more resources to increase the safety margin, for example, before the SLA is violated. In other words, cost function weights are used in optimization procedures or models that favor higher resource allocations to ensure that the SLA is not violated. That is, after the first run does not return a suitable candidate edge layer and edge platform placement (e.g., does not meet the SLA threshold, does not meet the power resource requirements, etc.), another run is performed with adjusted values ​​for the cost function and resource requirements. In some examples disclosed herein, any number of sequential iterations may be used until there are no more projected SLA violations, or projected latency and throughput deviate from the SLA, or the maximum number of iterations or time for sequential iterations is exceeded. Other examples may use different convergence flows in which the sequential iteration aspect described above is integrated into the trained model.

[0044] Examples disclosed herein can collect incoming telemetry and the expected future utilization of various nodes based on determined edge layer and edge platform placement decisions. Thus, current telemetry consists of both actual measurements (which may be slightly time-shifted into the past) and expected utilization from ongoing request placement decisions.

[0045] In some cases, if an edge service provider determines that sufficient resources are not available in advance at and / or near its preferred edge platform location, it may procure additional resources by paying a premium cost to access edge co-location ("Co-Lo") infrastructure.

[0046] In some examples disclosed herein, cached results are periodically aged out so that the system can adapt to changing conditions rather than being locked into suboptimal solutions, for example, as statistics on SLA violations improve or deteriorate.

[0047] In some examples, incoming requests and data from summarized telemetry are summarized along with the actual results of placement decisions (whether they meet or not SLA criteria) and uploaded to a backend cloud. These are used to train models that map workload types (i.e., request types) and quantized SLAs to performance, bandwidth, and power needs, or to refine such training. Similar offline training may be used, for example, to adjust default weights on a cost function.

[0048] The examples disclosed herein can be extended using AI-based techniques such as reinforcement learning. For example, generated data and post-mortem performance and telemetry data for each service can be provided to an AI model, which can then use the disclosed examples to have different inputs in its decision-making process. Like many other types of architectures, reinforcement learning can be implemented in a centralized location such as the cloud (and fed back to distributed layers), or in a federated learning approach. The latter may allow for more specific learning based, for example, on the geographical location or different parts of edge computing system 100.

[0049] As used herein, “edge platform” refers to an edge gateway platform, edge aggregation platform, fog platform (e.g., Cloudlet), core data center, and / or workload implemented / executed in an edge environment. In the examples disclosed herein, an edge platform can be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. In the examples disclosed herein, an edge platform distributes resources by making them available for allocation as compute assets assembled into virtual machines, pods, containers, etc., and by provisioning the assembled virtual machines, pods, containers, etc., for the execution of workloads through aggregation and networking.

[0050] As used herein, “workload” refers to an application task that runs on a cloudlet (for example, a collection of virtualized resources (processors, memory, storage, acceleration units) dynamically provisioned on one or more physical platforms) and has various computational, performance, security, and cost requirements.

[0051] As used herein, “edge node” refers to several (one, two, five, etc.) interconnected machines having processors, memory, etc., that are allocated to provision the resources required by the workload running on the edge node. In the examples disclosed herein, determining an edge node also refers to determining an edge tier (near edge, far edge, access edge, data center cloud, etc.) for running the workload.

[0052] As used herein, “edge tier placement” and “edge tier and edge platform placement” refer to the placement of workloads on edge platforms within an edge tier, and the selection of resources within the edge environment used to perform computational operations for a given workload.

[0053] As used herein, “optimal operating parameter percentage” means a function that determines the proportion of edge layer and edge platform placements that include several desirable operating parameters, such as i) available, ii) available and closest, iii) available, closest and cheapest, and iv) available, cheapest and sufficiently close.

[0054] Figure 1 shows an exemplary edge computing system 100 for providing edge services and applications to multi-stakeholder entities, distributed across layers of the edge computing system 100, including one or more client computing platforms 102, one or more edge gateway platforms 112, one or more edge aggregation platforms 122, one or more core data centers 132, and a global network cloud 142. Implementations of the edge computing system 100 may be provided by, or on behalf of, a telecommunications service provider ("telco" or "TSP"), an Internet of Things service provider, a cloud service provider (CSP), an enterprise entity, or any number of other entities. Various implementations and configurations of the edge computing system 100 may be provided dynamically, such as when orchestrated to meet service objectives.

[0055] Individual platforms or devices of the edge computing system 100 are located in specific layers corresponding to layers 120, 130, 140, 150, and 160. For example, client computing platforms 102a, 102b, 102c, 102d, 102e, and 102f are located in the endpoint layer 120, and edge gateway platforms 112a, 112b, and 112c are located in the edge device layer 130 (local level) of the edge computing system 100. Furthermore, edge aggregation platforms 122a and 122b (and / or fog platform 124 if it is placed or operated together with or between fog networking configuration 126) are located in the network access layer 140 (intermediate level). Fog computing (or "fogging") generally refers to the ability of cloud computing to extend an enterprise network to its edge, or to manage transactions across the cloud / edge landscape, typically in a coordinated, distributed network or multi-node network. Some forms of fog computing provide the deployment of compute, storage, and network services between end devices and cloud computing data centers, replacing cloud computing locations. Some forms of fog computing also provide the ability to manage workload / workflow-level services with respect to overall transactions by pushing certain workloads to the edge or cloud, based on the ability to meet overall service level agreements.

[0056] Fog computing, in many scenarios, provides a decentralized architecture and, by working with one or more edge node devices, functions as an extension of cloud computing, providing a greater and much greater amount of localized control, configuration, and management for the end devices. Furthermore, fog computing provides the ability for edge resources to collaborate to generate an edge-local cloud that can identify similar resources and be used, either alone or in conjunction with cloud computing, to complete compute, storage, or connectivity-related services. Fog computing can also allow cloud-based services to extend their reach to the edge of the network of devices, providing local and faster accessibility to edge devices. Thus, some forms of fog computing provide operation consistent with edge computing discussed herein, and the aspects of edge computing discussed herein are also applicable to fog networks, fogging, and fog configurations. Furthermore, the aspects of edge computing systems discussed herein may be configured as fog, or the aspects of fog may be integrated into an edge computing architecture.

[0057] The core data center 132 is located at the core network layer 150 (regional or geographically central level), while the global network cloud 142 is located at the cloud data center layer 160 (national or global layer). The use of “core” is provided as a term for a centralized network location deeper within the network, accessible by multiple edge platforms or components, but “core” does not necessarily indicate the “center” or deepest location of the network. Thus, the core data center 132 may be located within, in, or near the edge cloud 110. Figure 1 shows an exemplary number of client computing platforms 102a, 102b, 102c, 102d, 102e, 102f; edge gateway platforms 112a, 112b, 112c; edge aggregation platforms 122a, 122b; edge core data center 132; and global network cloud 142. However, it should be understood that the edge computing system 100 may include any number of devices and / or systems at each layer. Devices at any layer can be configured with respect to each other as peer nodes and / or peer platforms, and thus work collaboratively to achieve service objectives. For example, in an additional or alternative example, edge gateway platforms 112a, 112b, 112c can be configured as an edge of edges so that edge gateway platforms 112a, 112b, 112c communicate via peer-to-peer connections. In some examples, the edge aggregation platforms 122a, 122b and / or fog platform 124 can be configured as an edge of edges so that the edge aggregation platforms 122a, 122b and / or fog platform communicate over peer-to-peer connections.Furthermore, as shown in Figure 1, the number of components in each layer 120, 130, 140, 150, and 160 generally increases at each lower level (for example, as you move closer to the endpoints (e.g., client computing platforms 102a, 102b, 102c, 102d, 102e, 102f)). In this way, one edge gateway platform 112a, 112b, 112c can serve multiple platforms among the client computing platforms 102a, 102b, 102c, 102d, 102e, 102f, and one edge aggregation platform (e.g., one of the edge aggregation platforms 122a, 122b) can serve multiple platforms among the edge gateway platforms 112a, 112b, 112c.

[0058] In accordance with the examples provided herein, a client computing platform (for example, one of the client computing platforms 102a, 102b, 102c, 102d, 102e, or 102f) can be implemented as any type of endpoint component, device, instrument, or other capable of communicating as a data producer or consumer. For example, a client computing platform could include a mobile phone, laptop computer, desktop computer, or processor platform in an autonomous vehicle. In additional or alternative examples, a client computing platform could include a camera, sensor, etc. Furthermore, the labels “platform,” “node,” and / or “device” used in edge computing system 100 do not necessarily mean that such platforms, nodes, and / or devices operate in the role of clients or agents / minions / followers. Rather, any platform, node, and / or device in edge computing system 100 refers to individual entities, platforms, nodes, and / or subsystems containing discrete and / or connected hardware and / or software configurations for assisting and / or using edge cloud 110.

[0059] Thus, the edge cloud 110 is formed from network components and functional features operated within them by the edge gateway platforms 112a, 112b, and 112c, and the layer 130 and 140 edge aggregation platforms 122a and 122b, respectively. The edge cloud 110 can be implemented as any type of network providing edge computing and / or storage resources located in close proximity to radio access network (RAN) enabled endpoint devices (e.g., mobile computing devices, IoT devices, smart devices, etc.), which are shown in Figure 1 as client computing platforms 102a, 102b, 102c, 102d, 102e, and 102f. In other words, the edge cloud 110 may be conceived as an "edge" that connects endpoint devices and traditional network access points, while also providing storage and / or computing capabilities, and functioning as an entry point to service provider core networks, including mobile carrier networks (e.g., Global Mobile Communication System (GSM) networks, Long-Term Evolution (LTE) networks, 5G / 6G networks, etc.). Other types and forms of network access (e.g., Wi-Fi, long-range wireless including optical networks, wired networks) may also be used instead of or in combination with such 3GPP carrier networks.

[0060] In some examples, the edge cloud 110 may form part of, or otherwise provide, an entry point to, or across, a fog networking configuration 126 (for example, a network of one or more fog platforms 124; details not shown), which may be implemented as a system-level horizontal and distributed architecture for distributing resources. For example, the edge cloud 100 may form part of an entry point to the fog network configuration 126 by making resources or functions available for allocation as compute assets assembled into virtual machines, pods, containers, etc. Furthermore, the edge cloud 110 may provision the assembled virtual machines, pods, containers, etc., for performing workloads through aggregation and networking, as well as for services to perform specific functions. For example, the coordinated distributed network of fog platforms 124 may perform compute, storage, control, or networking aspects in the context of an IoT system configuration. Between the core data center 132 and client endpoints (e.g., client computing platforms 102a, 102b, 102c, 102d, 102e, 102f), other network-connected, aggregated, and distributed functions may exist in the edge cloud 110. Some of these, including the use of virtual edges and virtual services orchestrated for multiple tenants, are discussed in the following sections in the context of network functions or service virtualization.

[0061] As will be discussed in more detail below, the edge gateway platforms 112a, 112b, and 112c and the edge aggregation platforms 122a and 122b work together to provide various edge services and security to the client computing platforms 102a, 102b, 102c, 102d, 102e, and 102f. Furthermore, since each client computing platform (for example, one of the client computing platforms 102a, 102b, 102c, 102d, 102e, and 102f) can be fixed or mobile, each edge gateway platform 112a, 112b, and 112c can work with other edge gateway platforms to propagate currently provided edge services, associated service data, and security as the corresponding client computing platforms 102a, 102b, 102c, 102d, 102e, and 102f move around the area where they are located. To do so, the edge gateway platforms 112a, 112b, 112c, and / or edge aggregation platforms 122a, 122b can support multi-tenant and multi-tenant configurations, and services (or services hosted for them) from multiple service providers, owners, and multiple consumers can be supported and coordinated across one or more computing devices.

[0062] In the examples disclosed herein, the edge platform within the edge computing system 100 includes meta-orchestration capabilities. For example, an edge platform at a far edge (e.g., an edge platform closer to the edge user, edge device layer 130, etc.) can reduce the performance or power consumption of orchestration tasks associated with the far edge platform so that the execution of orchestration components on the far edge platform consumes only a small portion of the power and performance available on the far edge platform.

[0063] Orchestrators on various far-edge platforms participate in an end-to-end orchestration architecture. The examples disclosed herein anticipate that a comprehensive operating software framework (e.g., an open network automation platform, ONAP, or a similar platform) will be extended or options will be created within it, and therefore the examples disclosed herein may be compatible with those frameworks. For example, an orchestrator on an edge platform implementing the examples disclosed herein can interface with the ONAP orchestration flow and facilitate edge platform orchestration and telemetry activities. An orchestrator implementing the examples disclosed herein functions to coordinate orchestration and telemetry activities performed on an edge platform, such coordination including increasing or decreasing the power and / or resources consumed by local orchestration and telemetry components, delegating orchestration and telemetry processes to a remote computer, and / or taking back orchestration and telemetry processes from the remote computer when power and / or resources are available.

[0064] The remote devices described above are located in alternative locations relative to the edge platform, offloading the telemetry and orchestration processes. For example, the remote devices described above can, in contrast, be located on a near-edge platform (e.g., network access layer 140, core network layer 150, central office, mini data center, etc.). By offloading the telemetry and / or orchestration processes to a near-edge platform, the orchestrators on the near-edge platform are guaranteed (relatively) stable power supply and sufficient computing resources, which facilitates the execution of the telemetry and / or orchestration processes. An orchestrator on a near-edge platform (e.g., operating according to a global loop) can take over the telemetry and / or orchestration processes delegated from an orchestrator on a far-edge platform (e.g., operating according to a local loop). For example, if an orchestrator on a near-edge platform takes a delegated telemetry and / or orchestration process, the orchestrator on the near-edge platform can return the delegated telemetry and / or orchestration process to the orchestrator on the far-edge platform when conditions change on the far-edge platform (for example, when power and computing resources on the far-edge platform meet a threshold level, such as when a higher level of power and / or computing resources become available on the far-edge platform).

[0065] Within the Edge Cloud 110 architecture, diverse security approaches can be utilized. In a multi-stakeholder environment, there may be multiple loadable security modules (LSMs) used to provision policies that enforce the interests of stakeholders, including the tenant's interests. In some cases, other operators, service providers, etc., may have security interests that conflict with the tenant's interests. For example, a tenant may prefer to receive a full service for free (e.g., provided by the edge platform), while a service provider may want to receive full payment for doing little work or incurring little cost. An enforcement point environment can support multiple LSMs to which a combination of loaded LSM policies is applied (for example, the most restrictive and effective policy is applied, such as access being restricted if any of stakeholders A, B, or C restricts access). Within Edge Cloud 110, each edge entity can provision LSMs that enforce the interests of the edge entity. A cloud entity can provision LSMs that enforce the interests of the cloud entity. Similarly, various fog and IoT network entities can provision LSMs that enforce the interests of the fog entity.

[0066] In these examples, services can be thought of in terms of transactions performed for a set of contracts or ingredients, whether considered at the component level or at a human-perceptible level. Therefore, a user who has a service contract with a service provider expects the service to be delivered under the terms of the SLA. While not discussed in detail, the use of edge computing techniques discussed herein may play a role during contract negotiation and the measurement of contract performance (for example, in identifying what elements are required by the system to perform the service, and how the system responds to service conditions and changes).

[0067] Furthermore, in the examples disclosed herein, the edge platform and / or its orchestration components may take into account several factors when orchestrating services and / or applications in the edge environment. These factors include next-generation central office smart network function virtualization and service management, improving the performance per watt of the edge platform and / or orchestration components to overcome power limitations at the edge platform, reducing power consumption of the orchestration components and / or the edge platform, improving hardware utilization to enhance management and orchestration efficiency, providing physical and / or end-to-end security, providing quality of service and / or service level agreement satisfaction for individual tenants, improving the level of network equipment construction system compliance for each use case and tenant business model, pooling of acceleration components, billing and metering policies to improve the edge environment, improving the accuracy of results, reducing the likelihood and duration of temporary interruptions, and otherwise improving the perceived quality of the user experience.

[0068] "Service" is a broad term that often applies to a variety of situations, but generally refers to a relationship between two entities where one entity offers and performs work for the benefit of another entity. However, services provided by one entity to another must be performed with certain guidelines in place, which guarantee trust between the entities and manage transactions according to contractual terms defined at the start, during, and end of the service.

[0069] The following describes exemplary relationships between services used in edge computing systems. In edge computing scenarios, there are several services and interdependent, operating transaction layers. These services form a “service chain.” At the lowest level, ingredients constitute the system. These systems and / or resources communicate and cooperate with each other to provide numerous services to each other and to other persistent or transient entities in the surrounding environment. These entities can then provide services that people can consume. In this hierarchy, the services provided at each tier must be transactionally connected to ensure that the individual components (or sub-entities) providing the services adhere to contractually agreed-upon purposes and specifications. Deviations at any layer can have an overall impact on the entire service chain.

[0070] One type of service that can be provided in the hierarchical structure of edge environments is silicon-level services. For example, software-defined silicon (SDSi) type hardware provides the ability to ensure low-level compliance to transactions through its ability to intra-scale, manage, and guarantee the provision of functioning service-level agreements. The use of SDSi and similar hardware controls provides the ability to associate functions and resources within a system with specific tenants and to manage individual titles (rights) to those resources. The use of such functions is one way to dynamically "bring" computing resources into the workload.

[0071] For example, functional level agreements and / or service level agreements can define "transaction throughput" or "timeliness." In the case of SDSi, systems and / or resources can sign up to guarantee specific service level specifications (SLS) and service level objectives (SLOs) of a service level agreement (SLA). For example, an SLO might correspond to specific key performance indicators (KPIs) of an application (e.g., service, workload, etc.) (e.g., frames per second, floating-point operations per second, latency targets, etc.), and an SLA might correspond to a platform-level agreement to meet a specific SLO (e.g., 1 gigabyte of memory for 10 frames per second). SDSi hardware also provides infrastructure and resource owners with the ability to allow silicon components (e.g., components of a complex system that generates metric telemetry) to access and manage (add / remove) product functionality, and to freely scale up and down the functionality and utilization of the hardware. Furthermore, it provides the ability to provide deterministic feature assignment on a per-tenant basis. Furthermore, it provides the ability to link deterministic orchestration and service management to the dynamic (or subscription-based) activation of features without interrupting running services or client operations, or requiring a system reset or reboot.

[0072] At the lowest layer, SDSi can provide services and warranties to the system to ensure active compliance with contractually agreed-upon service level specifications that a single resource must provide within the system. Furthermore, SDSi provides the ability to manage the contractual rights (title), usage and associated finances of one or more tenants per component, as well as silicon-level functions (e.g., SKU functions). Silicon-level functions may relate to computation, storage or networking functions, performance, determinism, or even security, encryption, acceleration, etc. These functions not only enable tenants to achieve specific service level agreements but also assist with management and data collection, ensuring that transaction and contractual agreements are guaranteed at the lowest manageable component level.

[0073] At higher layers of the service hierarchy, Resource Level Services include systems and / or resources that provide the ability to meet workload demands (either entirely or through synthesis) by acquiring and enabling system-level functionality via SDSi, or through the synthesis of individually addressable resources (compute, storage, and network). At even higher layers of the service hierarchy, Workflow Level Services are horizontal, as service chains may have workflow-level requirements. Workflows describe dependencies between workloads to provide specific service-level objectives and requirements for end-to-end services. These services may include features and functions such as high availability, redundancy, recovery, fault tolerance, or load balancing (and many more). Workflow services define dependencies and relationships between resources and systems, describe requirements for relevant networks and storage, and describe transaction-level requirements and associated contracts to ensure end-to-end services. Workflow-level services are typically measured by service level objectives and have essential and expected service requirements.

[0074] At higher layers of the service hierarchy, Business Functional Services (BFS) are operational. These services are interrelated and represent different elements of the service that provide specific functionalities for the customer. In the case of edge computing, in the example of autonomous driving, a business function might constitute a service such as "timely arrival at an event"—this service would require several business functions that work together to achieve the user entity's goals: GPS guidance, Road Side Unit (RSU) awareness of local traffic conditions, user entity payment history, and user entity authorization of resources. Furthermore, since these BFS services multiple entities, each BFS manages its own SLA and recognizes its ability to address demands (workload and workflow) for its own resources. As requirements and demands increase, service change requirements are communicated to service entities at the workflow and resource levels, allowing those service entities to provide insights into their ability to meet those requirements. This step supports the overall transaction and service delivery to the next layer.

[0075] At the highest layer of the service hierarchy, Business Level Services (BLS) are tied to the functionality they provide. At this level, the customer or entity may not care how the service is structured or what components are used, managed, and / or tracked to deliver the service. The primary objective of Business Level Services is to achieve customer-defined goals in accordance with the overall contractual terms established between the customer and the provider in the agreed-upon financial agreement. BLS consists of several Business Functional Services (BFS) and an overall Service Level Agreement (SLA).

[0076] The configuration and other service management functions described herein are designed to meet the diverse requirements of edge computing through their unique and complex resource and service interactions. This service management configuration is intended to handle some of the fundamental resource services within the framework in an inherent way, rather than through the functions of agents or middleware. Services such as location, discovery, addressing, tracing, tracking, identification, and / or registration may be enabled as soon as a resource appears on the framework, and the manager or owner of a resource domain may use administrative rules and policies to ensure orderly resource discovery, registration, and authentication.

[0077] Furthermore, any number of edge computing architectures described herein may be adapted with service management capabilities. These capabilities may enable the system to constantly be aware of and record information regarding the movement, vectors, and / or direction of resources, and to fully describe these capabilities as both telemetry and metadata associated with the devices. These service management capabilities, as well as security elements, can be used for resource management, billing, and / or metering. The same capabilities apply to the associated resources, and less intelligent devices such as sensors may be attached to more manageable resources such as edge gateways. The service management framework is made aware of changes in the oversight or encapsulation of resources. Because nodes and components may be directly accessible or managed indirectly through a parent device or an alternative responsible device, either for a short period or throughout their lifecycle, this type of architecture is relayed to the service framework through its interface and made available to external query mechanisms.

[0078] Furthermore, this service management framework is always service-oriented, naturally balancing service delivery requirements with resource capabilities and availability, as well as access for data uploads to data analytics systems. If network transport degrades, fails, or changes to a higher-cost or lower-bandwidth capability, the service policy monitoring function provides alternative analytics and service delivery mechanisms within the bounds of user privacy or cost constraints. Using these functions, policies can trigger calls to analytics and dashboard services at the edge, ensuring continuous service availability, albeit with reduced fidelity or granularity. Once network transport is re-established, normal data collection, upload, and analytics services can resume.

[0079] The deployment of a multi-stakeholder edge computing system can be arranged and orchestrated to enable the deployment of multiple services and virtual edge instances across multiple edge platforms and subsystems for use by multiple tenants and service providers. In a system example applicable to a cloud service provider (CSP), the deployment of the edge computing system may be provided via an "over-the-top" approach to introduce the edge computing platform as a supplementary tool to cloud computing. In a contrasting system example applicable to a telecommunications service provider (TSP), the deployment of the edge computing system may be provided via a "network aggregation" approach, introducing the edge computing platform at a location where network access (from different types of data access networks) is aggregated. However, these over-the-top and network aggregation approaches may be implemented together in a hybrid or merged approach or configuration.

[0080] Figure 2 shows an exemplary implementation of an edge platform 200 for processing workload requests received from client compute nodes, as taught in this disclosure. For example, any of the edge gateway platforms 112a, 112b, 112c; edge aggregation platforms 122a, 122b; fog platform 124; and / or core data center 132 can be implemented by the edge platform 200. The exemplary edge platform 200 in Figure 2 includes an exemplary orchestrator 202, an exemplary function controller 204, an exemplary telemetry controller 206, an exemplary edge platform (EP) database 208, and an exemplary resource 210. In the example in Figure 2, any of the orchestrator 202, function controller 204, telemetry controller 206, EP database 208, and / or resource 210 can communicate via an exemplary communication bus 212. In the examples disclosed herein, the communication bus 212 may be implemented using any preferred wired and / or wireless communication. In additional or alternative examples, the communication bus 212 includes software, machine-readable instructions, and / or communication protocols through which information is communicated between the orchestrator 202, the function controller 204, the telemetry controller 206, the EP database 208, and / or the resources 210. In some examples, the orchestrator 202 is a means for orchestrating or orchestration means. In some examples, the function controller 204 is a means for controlling functions or function control means. In some examples, the telemetry controller 206 is a means for controlling telemetry or telemetry control means. In some examples, the EP database 208 is a means for storing or storage means. In some examples, the resources 210 are a means for resources or resource means.

[0081] In some examples, the edge platform 200 may be used in conjunction with Information Centric Networking (ICN), where the ICN operates as a layer on top of the existing edge network layer. For example, the ICN may be connected to the EP database 208 in the form of ICN routing nodes that cache and replicate the contents of the EP database 208 across multiple edge platforms 200. In some examples, the ICN optimizes access to the EP database 208 by bringing it closer to other EP nodes that request read / write access in the context of implementing the orchestrator 202, function controller 204, telemetry controller 206, and / or resource 210.

[0082] In the example shown in Figure 2, the orchestrator 202, function controller 204, telemetry controller 206, EP database 208, and resource 210 are included in, correspond to, and / or otherwise substitute for the edge platform 200. However, in some examples, one or more of the orchestrator 202, function controller 204, telemetry controller 206, EP database 208, and resource 210 may be included in an edge environment that includes the edge platform 200 (for example, edge cloud 110), rather than being included in the edge platform 200 itself. For example, the orchestrator 202 can connect to the endpoint layer (e.g., endpoint layer 120), the edge device layer (e.g., edge device layer 130), the network access layer (e.g., network access layer 140), the core network layer (e.g., core network layer 150), and / or the cloud data center layer (e.g., cloud data center layer 160) while outside the edge platform 200.

[0083] In other examples, one or more of the orchestrator 202, function controller 204, telemetry controller 206, EP database 208, and resource 210 are separate devices included in the edge environment. Furthermore, one or more of the orchestrator 202, function controller 204, telemetry controller 206, EP database 208, and resource 210 may be included in the edge device layer (e.g., edge device layer 130), the network access layer (e.g., network access layer 140), the core network layer (e.g., core network layer 150), and / or the cloud data center layer (e.g., cloud data center layer 160). For example, the orchestrator 202 may be included in the edge device layer (e.g., edge device layer 130), or the resource 210 may be included in the network access layer (e.g., network access layer 140), the core network layer (e.g., core network layer 150), and / or the cloud data center layer (e.g., cloud data center layer 160).

[0084] In some examples, in response to requests to perform workloads from client computing platforms (e.g., one of client computing platforms 102a, 102b, 102c, 102d, 102e, 102f) and / or edge platforms, orchestrator 202 parses the requests to determine operational parameters for the workloads from the client computing platforms (e.g., edge platforms). In some examples, orchestrator 202 may receive multiple requests and may deduplicate those requests. That is, orchestrator 202 may receive multiple requests to perform “workload A”. Instead of parsing all the requests for “workload A”, orchestrator 202 can save resources (e.g., thermal resources, computing resources, etc.) by parsing one of the requests for “workload A” and incorporating the result into the remaining requests for “workload A”.

[0085] To determine operational parameters, the example orchestrator 202 derives a subset of factors required for those parameters. In some examples, orchestrator 202 derives a subset of factors that include security levels, precision levels, dependencies on other microservices of the edge platform, location of dependencies within the edge environment, edge platform usage information, or service level agreement (SLA) attributes of the edge platform and / or workload. To conserve computational resources, orchestrator 202 can derive a subset of factors from a lookup table. For example, orchestrator 202 may receive a request for "Workload A", look up "Workload A" in a table, and determine a subset of factors that include service level agreement (SLA) attributes or a specific level of security required to perform "Workload A".

[0086] In some examples, when determining operating parameters, the orchestrator 202 estimates resource requirements for the edge platform and / or workload. Resource requirements include power measurements, thermal measurements, latency measurements, and / or bandwidth measurements for the workload, based on the aforementioned subset of factors. For example, the orchestrator 202 determines a subset of factors from a lookup table and estimates the resource requirements needed to satisfy that subset of factors. In the illustrated example, the orchestrator 202 updates the resource requirements based on the updated operating parameters or subset of factors.

[0087] In the illustrated example, the orchestrator 202 analyzes the candidate edge layer and edge platform placements for the request(s) For example, the function controller 204 can determine, based on function data, the resources allocated to the edge platform 200, such as hardware resources (e.g., hardware resources for computing, networking, security, and storage), software resources (e.g., firewalls, load balancers, virtual machines (VMs), guest operating systems (OS), applications, hypervisors, etc.), and / or combinations thereof, from which edge computing workloads (e.g., registered workloads) can be executed. In some examples, the function controller 204 can determine the containers that are provisioned and / or run on the edge platform 200. For example, the function controller 204 can identify the microservices associated with the containers provisioned on the edge platform 200 and / or the resources allocated to those containers on the edge platform 200.

[0088] In some examples, the function controller 204 retrieves function data from the EP database 208. For example, when the orchestrator 202 receives a request to execute a workload, the orchestrator 202 identifies whether the functionality of the edge platform 200 includes the appropriate resources(s) to fulfill the workload task by accessing the function controller 204 and / or the EP database 208. For example, if the orchestrator 202 receives a request to execute a workload that requires a processor with two cores, the orchestrator 202 can access the function controller 204 and / or the EP database 208 to determine whether the edge platform 200 includes the functionality to handle the requested workload. That is, the orchestrator 202 accesses the function controller 204 to determine resource availability for identifying candidate edge layer and edge platform placements for the request (e.g., the workload).

[0089] In the example in Figure 2, the function controller 204 further determines the functions of new and / or additional resources assigned to the edge platform 200. For example, if the edge platform 200 is upgraded by an edge service provider to include additional computing, storage, and / or network resources, the function controller 204 can register the additional resources and generate function data associated with them. In some examples, the function controller 204 can generate a protocol for interface with a resource (e.g., resource 210) on the edge platform 200 and / or send it to one or more of the orchestrator 202, telemetry controller 206, and / or EP database 208.

[0090] In the example illustrated in Figure 2, the telemetry controller 206 improves the distribution and execution of edge computing workloads (e.g., among the edge platforms) based on telemetry data related to edge platforms in the edge computing environment. For example, based on the telemetry data, the telemetry controller 206 can determine that a first edge platform and / or a second edge platform have one (or more) available resources 210, such as hardware resources (e.g., computing, networking, security, storage (e.g., non-volatile memory express)), software resources (e.g., firewalls, load balancers, virtual machines (VMs), guest operating systems (OS), applications, hypervisors, etc.), and / or combinations thereof, from which edge computing workloads can be executed. In such examples, telemetry data may include utilization rates (e.g., the percentage of resources being used or not used), delays in service (e.g., mean delay) (e.g., latency), the rate at which resources are available (e.g., average rate) (e.g., bandwidth, throughput, etc.), power consumption, temperature, etc., related to one (or more) of resources 210 of at least one of the edge platforms (e.g., edge platform 200 and / or alternative edge platforms).

[0091] Based on the placement of candidate edge layers and edge platforms that satisfy the operational parameters, the orchestrator 202 determines the operational parameter percentages for each candidate edge layer and edge platform placement to determine the placement of candidate edge layers and edge platforms for an edge platform (e.g., workload or cloud). In the illustrated example, the operational parameter percentages correspond to the percentage of resource availability for edge layer and edge platform placements that satisfy the resource requirements of the edge platform. For example, the orchestrator 202 may receive a request to run a workload that requires a processor with two cores. In this example, the orchestrator 202 can access the function controller 204 and / or the EP database 208 to determine the operational parameter percentages for each candidate edge layer and edge platform placement. In this example, the orchestrator 202 may determine that one candidate edge layer and edge platform configuration has a 100% operational parameter ratio (i.e., has a processor with two cores), and that the remaining candidate edge layer and edge platform configurations have a 0% operational parameter ratio (i.e., do not have a processor with two cores). The orchestrator 202 in the illustrated example selects the edge layer and edge platform configuration with the optimal operational parameter ratio as the candidate edge layer and edge platform configuration and performs the workload on that candidate edge layer and edge platform configuration. In some examples, the orchestrator 202 determines the optimal operational parameter ratio by combining several desirable operational parameters, such as i) available, ii) available and closest, iii) available, closest and cheapest, and iv) available, cheapest and sufficiently close, and determining the ratio associated with the edge layer and edge platform configuration that includes those desirable operational parameters.For example, the orchestrator 202 can determine that a certain candidate edge layer and edge platform configuration is the closest, available, and least expensive in the edge environment. Thus, the orchestrator 202 can determine an optimal ratio of operating parameters that is higher than other optimal ratios of operating parameters (e.g., edge layer configurations that are unavailable or more expensive).

[0092] In some examples, the orchestrator 202 adjusts resource requirements and cost function weights in response to the operational parameter percentages for each candidate edge layer and edge platform placement failing to meet thresholds (e.g., less than 70%, less than 40%). For example, the orchestrator 202 may adjust resource requirements by lowering operational parameter percentage thresholds or by reducing cost functions related to the amount that can be spent to perform the workload at a particular candidate edge layer and edge platform placement. However, after a number of successive iterations of threshold adjustments (e.g., resource requirements and cost function adjustments), the orchestrator 202 may determine that the request cannot be processed in the edge environment. For example, the cost associated with performing the workload fails to meet the cost function threshold. In some examples, the cost function is a penalty factor that reduces the optimal operational parameter percentage. For example, the cost function corresponds to penalties for exceeding statistical thresholds regarding latency, jitter, error rate, packet drop, network bandwidth consumption, etc.

[0093] In the illustrated example in Figure 2, the edge platform 200 includes an EP database 208 for recording data (e.g., telemetry data, workload, functional data, resource availability, etc.). The EP database 208 can be implemented by volatile memory (e.g., synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), RAMBUS dynamic random access memory (RDRAM), etc.) and / or non-volatile memory (e.g., flash memory). The EP database 208 can be additionally or alternatively implemented by double data rate (DDR) memory such as DDR, DDR2, DDR3, DDR4, and mobile DDR (mDDR). The EP database 208 can be additionally or alternatively implemented by one or more mass storage devices such as hard disk drives, compact disk drives, digital utility disk drives, and solid-state disk drives. In the illustrated example, the EP database 208 is shown as a single database, but the EP database 208 can be implemented by any number and / or types of databases. Furthermore, the data stored in the EP database 208 can be in any data format, such as binary data, comma-separated data, tab-separated data, or Structured Query Language (SQL) structures.

[0094] In the illustrated example in Figure 2, resource(s) 210 are invoked to execute workloads (e.g., edge computing workloads) derived from the client computing platform. For example, resource(s) 210 can correspond to and / or represent an edge platform or a part thereof. For example, the orchestrator(s) 202, function controller(s) 204, telemetry controller(s) 206, EP database(s) 208, and / or more generally, the edge platform(s) 200 can invoke one of each of resource(s) 210 to execute one or more edge computing workloads.

[0095] In some examples, resource 210 represents hardware resources, virtualizations of hardware resources, software resources, virtualizations of software resources, and / or combinations thereof. For example, resource 210 may include, and / or represent, one or more CPUs (e.g., multi-core CPUs), one or more FPGAs, one or more GPUs, one or more network interface cards (NICs), one or more vision processing units (VPUs), and / or any other type of hardware or hardware accelerator. In such examples, resource 210 may include, and / or represent, virtualizations of the one or more CPUs, one or more FPGAs, one or more GPUs, one or more NICs, and so on. In other examples, the orchestrator 202, function controller 204, telemetry controller 206, EP database 208, resource 210, and / or, more generally, the edge platform 200 may represent them in corresponding and / or other ways, including one or more software resources, virtualization of such software resources, such as a hypervisor, load balancer, OS, VM, and / or a combination thereof.

[0096] In some examples, in response to a request to execute a workload from a client computing platform (e.g., one of client computing platforms 102a, 102b, 102c, 102d, 102e, or 102f), the orchestrator 202 communicates with resource 210 and at least one of the client computing platforms (e.g., one of client computing platforms 102a, 102b, 102c, 102d, 102e, or 102f) to create a contract (e.g., a workload contract) associated with a description of the workload to be executed. The client computing platform (e.g., one of client computing platforms 102a, 102b, 102c, 102d, 102e, or 102f) provides the orchestrator 202 with tasks related to the contract and the workload description, and the orchestrator 202 schedules these tasks to be executed on the edge platform. The tasks may include the contract and the description of the workload to be executed. In some examples, a task involves a request to acquire and / or otherwise allocate resources used to perform the workload.

[0097] Figure 2 shows an exemplary way of implementing the edge platform 200 of Figure 1, but one or more of the elements, processes and / or devices shown in Figure 2 may be combined, divided, rearranged, omitted, removed, and / or implemented in any other way. Furthermore, the exemplary orchestrator 202, exemplary function controller 204, exemplary telemetry controller 206, exemplary EP database 208, exemplary resource 210, and / or more generally, the exemplary edge platform 200 of Figure 2 may be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. Therefore, for example, the exemplary orchestrator 202, the exemplary function controller 204, the exemplary telemetry controller 206, the exemplary EP database 208, the exemplary resource 210, and / or more generally, any of the exemplary edge platform 200 may be implemented by one or more analog or digital circuits, logic circuits, programmable processors, programmable controllers, graphics processing units (GPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable logic devices (PLDs), and / or field-programmable logic devices (FPLDs). If any of the apparatus or system claims of this patent are read to cover purely software and / or firmware implementations, then at least one of the exemplary orchestrator 202, exemplary function controller 204, exemplary telemetry controller 206, exemplary EP database 208, exemplary resource 210, and / or more generally exemplary edge platform 200 is expressly defined herein to include a memory, digital versatile disc (DVD), compact disc (CD), Blu-ray disc, or other non-temporary computer-readable storage device or storage disk containing the software and / or firmware.Furthermore, the exemplary edge platform 200 in Figure 2 may include, in addition to or instead of, one or more elements, processes, and / or devices shown in Figure 2, and / or two or more of any or all of the illustrated elements, processes, and devices. As used herein, the phrase “communicating” includes, including its variations, direct communication and / or indirect communication through one or more intermediate components, and does not require direct physical (e.g., wired) communication and / or constant communication, but rather further includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and / or one-off events.

[0098] Flowcharts representing exemplary hardware logic, machine-readable instructions, a hardware-implemented state machine, and / or any combination thereof for implementing the edge platform 200 are shown in Figures 3 to 6. The machine-readable instructions may be one or more executable programs or parts of executable programs for execution by a computer processor and / or processor circuitry, such as the processor 712 shown in the exemplary processor platform 700 described later in relation to Figure 7. The program may be embodied in software stored on a non-temporary computer-readable storage medium such as a CD-ROM, floppy disk, hard drive, DVD, Blu-ray disk, or memory associated with the processor 712; however, the entire program and / or parts thereof may alternatively be executed by a device other than the processor 712 and / or embodied in firmware or dedicated hardware. Furthermore, while the exemplary program is described with reference to the flowcharts shown in Figures 3 to 6, many other methods for implementing the exemplary edge platform 200 may be used alternatively. For example, the execution order of blocks may be changed, and / or some of the described blocks may be modified, deleted, or combined. Additionally or alternatively, any or all of the blocks may be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured to perform the corresponding operations without running software or firmware. The processor circuits may be distributed across different network locations and / or local to one or more devices (e.g., a multi-core processor in a single machine, multiple processors distributed across server racks, etc.).

[0099] The machine-readable instructions described herein may be stored in one or more of the following formats: compressed format, encrypted format, fragmented format, compiled format, executable format, packaged format, etc. The machine-readable instructions described herein may be stored as data or data structures (e.g., parts of instructions, code, representations of code, etc.) that can be used to create, manufacture, and / or generate machine-executable instructions. For example, machine-readable instructions may be fragmented and stored in one or more storage devices and / or computing devices (e.g., servers) located at the same or different locations in a network or set of networks (e.g., in the cloud, in edge devices, etc.). Machine-readable instructions may require one or more of the following processes to be directly readable, interpretable, and / or executable by computing devices and / or other machines: installation, modification, adaptation, updating, combination, supplementation, configuration, decryption, decompression, depacking, distribution, reallocation, compilation, etc. For example, machine-readable instructions may be stored in multiple parts, each individually compressed, encrypted, and stored in separate computing devices, which, when decrypted, decompressed, and combined, form a set of executable instructions that implement one or more functions, which together can form a program as described herein.

[0100] In another example, machine-readable instructions may be stored in a state where they can be read by processor circuitry, but execution of such instructions on a particular computing device or other device may require additional libraries (e.g., dynamic link libraries (DLLs)), software development kits (SDKs), application programming interfaces (APIs), etc. In yet another example, machine-readable instructions may need to be configured (e.g., settings are stored, data is entered, network addresses are recorded, etc.) before all or part of the machine-readable instructions and / or corresponding program can be executed. Thus, machine-readable media as used herein may contain machine-readable instructions and / or programs, regardless of the specific format or state of such machine-readable instructions and / or programs when stored, or otherwise stationary, or in transit.

[0101] The machine-readable instructions described herein can be expressed in any instruction language, scripting language, programming language, etc., of the past, present, or future. For example, machine-readable instructions may be expressed using any of the following languages: C, C++, Java, C#, Perl, Python, JavaScript, hypertext markup language (HTML), structured query language (SQL), Swift, etc.

[0102] As described above, the exemplary process in Figure 7 may be carried out using executable instructions (e.g., computer and / or machine-readable instructions) stored on non-temporary computer and / or machine-readable media such as hard disk drives, flash memory, read-only memory, compact disks, digital multipurpose disks, caches, random access memory, and / or other storage devices or storage disks where information is stored for any period of time (e.g., for a long period, permanently, for short instances, for temporary buffering, and / or for caching information). As used herein, the term non-temporary computer-readable media is explicitly defined as including any type of computer-readable storage device and / or storage disk, excluding propagating signals and excluding transmission media.

[0103] The terms “contain” and “have” (and all their forms and tenses) are used herein as open-ended terms. Thus, whenever a claim uses either “contain” or “have” (e.g., have, contain, possess, include, have, etc.) as a preamble or within any type of claim statement, it is understood that additional elements, items, etc., may exist without exceeding the scope of the corresponding claim or statement. Where used herein, for example, when the phrase “at least” is used as a transitional term in the preamble of a claim, it is open-ended, just as the terms “have” and “contain” are open-ended. The term “and / or” when used in the form A, B, and / or C, for example, refers to any combination or subset of A, B, and C, such as (1) A only, (2) B only, (3) C only, (4) A and B, (5) A and C, (6) B and C, and (7) A, B, and C. As used herein, in the context of describing structures, components, items, objects and / or things, the phrase “at least one of A and B” is intended to refer to an implementation that includes (1) at least one A, (2) at least one B, and (3) any of at least one A and at least one B. Similarly, as used herein, in the context of describing structures, components, items, objects and / or things, the phrase “at least one of A or B” is intended to refer to an implementation that includes (1) at least one A, (2) at least one B, and (3) any of at least one A and at least one B. As used herein, in the context of describing the execution or implementation of processes, instructions, actions, activities and / or steps, the term “at least one of A and B” is intended to refer to an implementation that includes (1) at least one A, (2) at least one B, and (3) any of at least one A and at least one B.Similarly, as used herein, in the context of describing the execution or implementation of a process, instruction, action, activity and / or step, the term “at least one of A or B” is intended to mean an implementation that includes (1) at least one A, (2) at least one B, and (3) any one of at least one A and at least one B.

[0104] Where used herein, singular references (e.g., “a,” “an,” “first,” “second,” etc.) do not exclude plurals. The term “a” or “an” entity, where used herein, refers to one or more of those entities. The terms “a” (or “an”), “one or more,” and “at least one” are interchangeable herein. Furthermore, multiple means, elements, or method actions, although individually enumerated, may be implemented, for example, by a single unit or processor. In addition, individual features may be included in different examples or claims, but these may be combined, and their inclusion in different examples or claims does not imply that the combination of features is unfeasible and / or unfavorable.

[0105] Figure 3 is a flowchart representing an exemplary machine-readable instruction 300 that may be executed to implement the exemplary edge platform 200 of Figure 2. The machine-readable instruction 300 begins in block 302, where the orchestrator 202 receives a request to execute a workload from the edge platform in the edge environment.

[0106] In block 304, the orchestrator 202 analyzes the request to determine the operational parameters for the workload from the edge platform. For example, the orchestrator 202 derives a subset of the factors required for the operational parameters.

[0107] In block 306, the orchestrator analyzes candidate edge layer and edge platform placements based on operational parameters. For example, the illustrated orchestrator 202 analyzes candidate edge layer and edge platform placements for the said request(s) based on operational parameters by determining the resource availability of each candidate edge layer and edge platform placement.

[0108] In block 308, the orchestrator determines candidate edge layer and edge platform configurations for a workload based on candidate edge layer and edge platform configurations that satisfy the workload requirements and operational parameters. For example, orchestrator 202 determines operational parameter percentages for each of the candidate edge layer and edge platform configurations. In the illustrated example, the operational parameter percentages correspond to the percentage of resource availability for the edge layer and edge platform configurations that meet the resource requirements of the workload. For example, orchestrator 202 may receive a request to run a workload that requires a processor with two cores. In this example, orchestrator 202 can access the function controller 204 and / or the EP database 208 to determine operational parameter percentages for each of the candidate edge layer and edge platform configurations. In this example, the orchestrator 202 can determine that one candidate edge layer and edge platform configuration has a 100% operational parameter ratio (i.e., has a processor with two cores), and that the remaining candidate edge layer and edge platform configurations have a 0% operational parameter ratio (i.e., do not have a processor with two cores). The illustrated example orchestrator 202 selects the edge layer and edge platform configuration with the optimal operational parameter ratio as the candidate edge layer and edge platform configuration and implements the workload in that configuration. In some examples, in response to the operational parameter ratio for each candidate edge layer and edge platform configuration not meeting a threshold (e.g., less than 70%, less than 40%), the orchestrator 202 adjusts the resource requirements and cost function weights.For example, the orchestrator 202 may adjust resource requirements by lowering operational parameter percentage thresholds, reducing cost functions related to how much money can be spent, how much data loss can be tolerated, or how much latency can be endured to perform a workload at a particular candidate edge layer and edge platform placement. However, after a threshold number of iterations (e.g., resource request and cost function adjustments), the orchestrator 202 may determine that the request cannot be processed in the edge environment. For example, the costs associated with performing the workload do not meet the cost function threshold. The machine-readable instruction 300 terminates.

[0109] Figure 4 is a flowchart representing an exemplary machine-readable instruction 304 that can be executed to implement the exemplary orchestrator 202 in Figure 2 to parse a request to determine operational parameters for a workload from an edge platform. The machine-readable instruction 304 begins in block 402, where the orchestrator 202 derives a subset of factors required for the operational parameters. For example, the orchestrator 202 may derive a subset of factors including the level of security, the level of precision, the workload's dependencies on other microservices, the location of those dependencies within the edge environment, usage information for the workload, or service level agreement (SLA) attributes for the workload. The orchestrator 202 may derive a subset of factors from a lookup table to conserve computational resources. For example, the orchestrator 202 may receive a request for "workload A", look up "workload A" in a table, and determine a subset of factors including a specific level of security or service level agreement (SLA) attributes required to perform "workload A".

[0110] In block 404, the orchestrator 202 estimates the resource requirements for the workload based on a subset of factors. For example, it estimates the resource requirements for the workload, including power measurements, thermal measurements, latency measurements, and / or bandwidth measurements for each edge platform used to process the workload, based on the subset of factors. For example, the orchestrator 202 determines a subset of factors from a lookup table and estimates the resource requirements needed to satisfy that subset of factors.

[0111] In block 406, the orchestrator 202 determines whether updated operating parameters are available. For example, the orchestrator 202 in the example shown updates resource requirements based on updated operating parameters or a subset of factors. If the orchestrator identifies that updated operating parameters are available, the machine-readable instruction 304 proceeds to block 402 to derive a subset of factors required for those operating parameters. For example, the orchestrator 202 may identify updated SLA attributes and proceed to derive an updated subset of factors required for the updated SLA attributes. If the orchestrator 202 determines that there are no available updated operating parameters, the machine-readable instruction 304 returns to block 306.

[0112] Figure 5 is a flowchart of an exemplary machine-readable instruction 306 that can be executed to implement the exemplary orchestrator 202 in Figure 2 to analyze candidate edge layer and edge platform placements based on operating parameters. The machine-readable instruction 306 begins in block 502, where the orchestrator 202 receives telemetry data from each of the candidate edge layer and edge platform placements. For example, the orchestrator 202 may receive telemetry data corresponding to the expected future use of various nodes based on the determined edge layer and edge platform placement decisions, actual measurements (which may be slightly time-shifted into the past), and expected use from the ongoing request placement decision.

[0113] In block 504, the orchestrator 202 determines the resource availability for each candidate edge layer and edge platform deployment. For example, resource availability corresponds to power availability, thermal availability, latency availability, and / or bandwidth availability for each candidate edge layer and edge platform deployment, based on telemetry data from each of them. In some examples, the orchestrator 202 communicates with the function controller 204 to determine resource availability.

[0114] In block 506, the orchestrator 202 determines whether updated telemetry data is available. If the orchestrator 202 determines that updated telemetry data is available, the machine-readable instruction 306 proceeds to block 502 and receives telemetry data from each of the candidate edge layers and edge platform placements. If the orchestrator 202 determines that updated telemetry data is not available, the machine-readable instruction 304 returns and proceeds to block 308.

[0115] Figure 6 is a flowchart representing exemplary machine-readable instructions 308 that may be executed to implement the exemplary orchestrator 202 in Figure 2 to determine candidate edge layer and edge platform configurations for a workload based on candidate edge layer and edge platform configurations that satisfy workload requirements and operational parameters. The machine-readable instructions 308 begin in block 602, where the orchestrator 202 determines operational percentages for each of the candidate edge layer and edge platform configurations. For example, the orchestrator 202 determines operational parameter percentages for each of the candidate edge layer and edge platform configurations. The operational parameter percentages in the example shown correspond to the percentage of resource availability for edge layer and edge platform configurations that satisfy the resource requirements of the edge platform. For example, the orchestrator 202 may receive a request to perform a workload that requires a processor with two cores. In this example, the orchestrator 202 may access the function controller 204 and / or the EP database 208 to determine operational parameter percentages for each of the candidate edge layer and edge platform configurations. In this example, the orchestrator 202 can determine that one candidate edge layer and edge platform configuration has a 100% operational parameter ratio (i.e., has a processor with two cores), and that the remaining candidate edge layer and edge platform configurations have a 0% operational parameter ratio (i.e., do not have a processor with two cores). In some examples, the orchestrator 202 may determine that a particular candidate edge layer and edge platform configuration is the closest, available, and least expensive in the edge environment. Thus, the orchestrator 202 may determine an optimal operational parameter ratio that is higher than other optimal operational parameter ratios (e.g., edge layer configurations that are unavailable, more expensive, etc.).

[0116] In block 604, the orchestrator 202 selects an edge layer and edge platform arrangement with the optimal ratio of operating parameters as a candidate edge layer and edge platform arrangement.

[0117] In block 606, the orchestrator 202 determines whether the operational parameter ratios meet the thresholds. For example, the illustrated orchestrator 202 selects edge layer and edge platform placements with optimal operational parameter ratios (e.g., 70%, the largest aggregate of the desired operational parameters, etc.) and compares these operational parameter ratios to thresholds (e.g., 75%, 80%, 90%, etc.). If the orchestrator 202 determines that the operational parameter ratios meet the thresholds, the machine-readable instruction 308 proceeds to block 608, where the orchestrator 202 performs the workload on the candidate layer edge placements.

[0118] If the orchestrator 202 determines that the operational parameter ratio does not satisfy a threshold, the machine-readable instruction 308 proceeds to block 610 to determine whether the number of iterations is below the threshold. For example, each time a request is received and processed by the orchestrator 202 to determine edge layer and edge platform placements, an iteration count is determined. For example, during a first iteration, the orchestrator 202 may receive and process a first request and determine candidate edge layer and edge platform placements. Thus, the request includes an iteration count of 1. That is, during the first iteration, the orchestrator 202 was able to determine edge layer and edge platform placements that satisfy the SLA attributes of the request. If the orchestrator 202 determines that the number of iterations is not below a threshold (e.g., 5, 8, 12, etc.), the machine-readable instruction 308 terminates. For example, the number of iterations may have exceeded the threshold because the cost associated with executing the workload exceeded the cost function threshold (e.g., the SLA attributes were not met).

[0119] If the orchestrator 202 determines that the number of iterations is below a threshold, the machine-readable instruction 308 proceeds to block 612 and increments the iteration count.

[0120] In block 614, the orchestrator 202 adjusts resource requirements and cost function weights. For example, the orchestrator 202 may adjust resource requirements by lowering operational parameter percentage thresholds and increase cost functions related to the amount that can be spent to perform the workload at a particular candidate edge layer and edge platform placement. In some examples, the orchestrator 202 increases the amount that can be spent to perform the workload at a candidate edge layer and edge platform placement. For example, the orchestrator 202 can update the cost function weights in block 614 (e.g., increase the amount that can be spent, increase the amount of resources available, increase the amount of edge platforms available, etc.) and return to block 602. The machine-readable instruction 308 returns to the machine-readable instruction 300 after performing the workload, or terminates when the number of iterations exceeds the limit.

[0121] Figure 7 is a block diagram of an exemplary processor platform 700 configured to implement the edge platform 200 by executing the instructions in Figures 3 to 6. The processor platform 700 could be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), a mobile device (e.g., a mobile phone, a smartphone, a tablet such as iPad®), a personal digital assistant (PDA), an internet appliance, a DVD player, a CD player, a digital video recorder, a Blu-ray player, a game console, a personal video recorder, a set-top box, a headset, or other wearable device, or any other type of computing device.

[0122] The illustrated example processor platform 700 includes a processor 712. The illustrated example processor 712 is hardware. For example, the processor 712 can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers of any desired family or manufacturer. The hardware processor may be a semiconductor-based (e.g., silicon-based) device. In this example, the processor implements an exemplary orchestrator 202, an exemplary function controller 204, an exemplary telemetry controller 206, an exemplary EP database 208, an exemplary resource 210, and / or more generally, an exemplary edge platform 200.

[0123] The processor 712 in the illustrated example includes local memory 713 (e.g., cache). The processor 712 in the illustrated example communicates with main memory, which includes volatile memory 714 and non-volatile memory 716, via bus 718. The volatile memory 714 may be implemented by synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), RAMBUS® dynamic random access memory (RDRAM®), and / or any other type of random access memory device. The non-volatile memory 716 may be implemented by flash memory and / or any other desired type of memory device. Access to main memory 714, 716 is controlled by a memory controller.

[0124] The illustrated example processor platform 700 also includes an interface circuit 720. The interface circuit 720 can be implemented by any type of interface standard, such as an Ethernet interface, Universal Serial Bus (USB), Bluetooth® interface, Near Field Communication (NFC) interface, and / or PCI Express interface.

[0125] In the illustrated example, one or more input devices 722 are connected to the interface circuit 720. The input devices 722 allow the user to input data and / or commands to the processor 712. The input devices can be implemented, for example, by audio sensors, microphones, cameras (still or video), keyboards, buttons, mice, touchscreens, trackpads, trackballs, IsoPoint, and / or voice recognition systems.

[0126] One or more output devices 724 are also connected to the interface circuit 720 of the illustrated example. The output devices 724 can be implemented, for example, by display devices (e.g., light-emitting diodes (LEDs), organic light-emitting diodes (OLEDs), liquid crystal displays (LCDs), cathode ray tube displays (CRTs), in-place switching (IPS) displays, touchscreens, etc.), haptic output devices, printers, and / or speakers. Thus, the interface circuit 720 of the illustrated example typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0127] The illustrated example interface circuit 720 also includes communication devices such as transmitters, receivers, transceivers, modems, residential gateways, wireless access points, and / or network interfaces to facilitate data exchange with external machines (e.g., any type of computing device) via the network 726. Communication can be conducted via, for example, Ethernet connections, digital subscriber line (DSL) connections, telephone line connections, coaxial cable systems, satellite systems, line-of-site wireless systems, cellular telephone systems, etc.

[0128] The illustrated example processor platform 700 also includes one or more mass storage devices 728 for storing software and / or data. Examples of such mass storage devices 728 include floppy disk drives, hard drive disks, compact disk drives, Blu-ray disk drives, redundant array (RAID) systems of independent disks, and digital multipurpose disk drives.

[0129] The machine-executable instructions 732 in Figures 3 to 6 may be stored in mass storage device 728, volatile memory 714, non-volatile memory 716, and / or on a removable non-temporary computer-readable storage medium such as a CD or DVD.

[0130] Figure 8 shows a block diagram illustrating an exemplary software distribution platform 805 for distributing software such as the exemplary computer-readable instruction 732 in Figure 7 to third parties. The exemplary software distribution platform 805 may be implemented by any computer server, data facility, cloud service, etc., that can store the software and transmit it to other computing devices. The third party may be a customer of the entity that owns and / or operates the software distribution platform. For example, the entity that owns and / or operates the software distribution platform may be the developer, seller, and / or licensor of the software such as the exemplary computer-readable instruction 732 in Figure 7. The third party may be a consumer, user, retailer, OEM, etc., who purchases and / or licenses the software for use and / or resale and / or sublicensing. In the illustrated example, the software distribution platform 805 includes one or more servers and one or more storage devices. The storage devices store the computer-readable instruction 732, which may correspond to the exemplary computer-readable instruction 300 in Figures 3 to 6, as described above. The one or more servers of the exemplary software distribution platform 805 communicate with a network 810, which may correspond to the Internet and / or one or more of the exemplary networks described above. In some examples, the one or more servers respond to requests to send software to requesters as part of a commercial transaction. Payment for the distribution, sale, and / or licensing of the software may be handled by the one or more servers of the software distribution platform and / or by a third-party payment agency. The servers enable purchasers and / or licensors to download computer-readable instructions 732 from the software distribution platform 805.For example, software that can correspond to the exemplary computer-readable instruction 732 in Figure 7 may be downloaded to an exemplary processor platform 1000, which executes the computer-readable instruction 732 to implement the edge platform 200. In some examples, one or more servers of the software distribution platform 805 periodically provide, transmit, and / or enforce updates to the software (e.g., the exemplary computer-readable instruction 732 in Figure 7) to ensure that improvements, patches, updates, etc., are distributed and applied to the software on end-user devices.

[0131] Figure 9 is a block diagram 900 showing an overview of the configuration for edge computing, including the processing layer referred to as “edge cloud” in many of the following examples. As illustrated, the edge cloud 910 is co-located at edge locations such as access points or base stations 940, local processing hubs 950, or central offices 920, and thus can include multiple entities, devices, and equipment instances. The edge cloud 910 is located much closer to endpoint (consumer and producer) data sources 960 (e.g., autonomous vehicles 961, user equipment 962, business and industrial equipment 963, video capture devices 964, drones 965, smart city and building equipment 966, sensors and IoT devices 967, etc.) than the cloud data center 930. The compute, memory, and storage resources provided at the edge within the edge cloud 910 are crucial for providing ultra-low latency response times for services and functions used by the endpoint data source 960, as well as for reducing network backhaul traffic from the edge cloud 910 to the cloud data center 930, thereby improving energy consumption and overall network utilization, among other benefits. In some examples, the edge platform 200 can be implemented within the edge cloud 910. In some examples, the edge platform 200 described above in relation to Figures 1-8 can be implemented as an access point or base station 940, a local processing hub 950, a central office 920, a data source 960 (e.g., autonomous vehicles 961, user equipment 962, business and industrial equipment 963, video capture devices 964, drones 965, smart city and building equipment 966, sensors and IoT devices 967, etc.), or a cloud data center 930. In some examples, the edge platform 200 can be implemented using the examples disclosed in Figures 9-15B.

[0132] Computation, memory, and storage are scarce resources and generally decrease with edge location (for example, fewer processing resources are available at base stations than at central offices, and fewer at consumer endpoint devices than at base stations). However, the closer the edge location is to the endpoint (e.g., UE), the more space and power constraints are often imposed. Therefore, edge computing attempts to reduce the amount of resources required for network services by allocating more resources that are closer, both geographically and in terms of network access time. In this way, edge computing attempts to move computing resources to where the workload data is located, or workload data to where the computing resources are located, as appropriate.

[0133] The following describes aspects of edge cloud architectures that cover multiple potential deployments and address constraints that some network operators and service providers may have in their own infrastructure. These include configuration variations based on edge location (for example, edges at the base station level may have more constrained performance and functionality in multi-tenant scenarios); configurations based on the types of compute, memory, storage, fabric, acceleration, or similar resources available to edge locations, location layers, or groups of locations; service, security, and management and orchestration capabilities; and variations for related objectives to achieve end-service usability and performance. These deployments can achieve processing in network layers that can be considered “near edge,” “close edge,” “local edge,” “intermediate edge,” or “far edge” layers, depending on latency, distance, and timing characteristics.

[0134] Edge computing is an evolving paradigm in which computing is performed through the use of computing platforms (e.g., x86 or ARM computing hardware architectures) implemented at the “edge” of the network, or closer to the “edge,” typically in base stations, gateways, network routers, or other devices much closer to endpoint devices that generate and consume data (e.g., in “local edge,” “close edge,” or “near edge”). For example, an edge gateway server may have a pool of memory and storage resources to perform real-time computations for low-latency use cases for connected client devices (e.g., autonomous driving or video surveillance). Alternatively, as an example, a base station may be augmented with computing and acceleration resources to directly handle service workloads for connected user equipment without further data transmission over a backhaul network. Or, as another example, central office network management hardware may be replaced with standardized computing hardware that performs virtualized network functions and provides computing resources for performing service and consumer functions for connected devices. Within edge computing networks, there can be scenarios where computing resources are "moved" to the data, and scenarios where data is "moved" to the computing resources. Alternatively, for example, base station computing, acceleration, and network resources can provide services to scale workload demands as needed by activating dormant capacity (on-demand capacity, subscriptions). This is to manage corner cases, emergencies, or to provide extended lifespan for deployed resources over a considerably longer implementation lifecycle.

[0135] Figure 10 illustrates the operational layers between endpoints, edge cloud, and cloud computing environments. Specifically, Figure 10 shows an example of compute use case 1005 that utilizes the edge cloud 910 across multiple exemplary layers of network computing. These layers begin with the endpoint (devices and things) layer 1000, which accesses the edge cloud 910 to perform data creation, analysis, and data consumption activities. The edge cloud 910 may span multiple network layers, for example, an edge device layer 1010 having gateways, on-premises servers, or network equipment (nodes 1015) located within physically close edge systems; a network access layer 1020 including base stations, radio processing units, network hubs, regional data centers, or local network equipment (devices 1025); and any devices, equipment, or nodes (layer 1012, not shown in detail) located between them. Network communication within the edge cloud 910 and between the various layers may occur via any number of wired or wireless media, including through connectivity architectures and technologies not shown.

[0136] Examples of latency resulting from network communication distance and processing time constraints include less than milliseconds (ms) within the endpoint layer 1000, less than 5 milliseconds at the edge device layer 1010 (e.g., the “near edge” or “close edge” layer), and in the range of 10 to 40 milliseconds when communicating with nodes in the network access layer 1020 (e.g., the “intermediate edge” layer). Outside the edge cloud 910 are the core network 1030 and cloud data center 1040 layers, each with increasing latency (e.g., between 50 and 60 ms at the core network layer 1030, and over 100 ms at the cloud data center layer, both of which can be considered “far edge” layers). As a result, operations in the core network data center 1035 or cloud data center 1045 with latency of at least 50 to 100 ms or more cannot achieve many of the time-critical functions of use case 1005. Each of these latency values ​​is provided for illustrative and contrast purposes only, and it should be understood that the use of other access network media and technologies may further reduce latency.

[0137] Various use cases 1005 involve multiple services utilizing the edge cloud, which may access resources under usage pressure from incoming streams. To achieve results with low latency, services running within the edge cloud 910 must balance various requirements regarding: (a) priority (throughput or latency) and quality of service (QoS) (e.g., traffic for an autonomous vehicle may have a higher priority than a temperature sensor in terms of response time requirements; or, depending on the application, there may be performance sensitivities / bottlenecks in compute / accelerator, memory, storage, or network resources); (b) reliability and resilience (e.g., some input streams must operate with mission-critical reliability and traffic must be routed accordingly, while some other input streams, depending on the application, may tolerate occasional failures); and (c) physical constraints (e.g., power, cooling, and shape factors).

[0138] The end-to-end service view of these use cases includes the concept of service flow and is associated with transactions. A transaction details the overall service requirements for the entities consuming the service and the associated services for resources, workloads, workflows, business functions, and business-level requirements. Services performed under the stated “conditions” may be managed at each layer to ensure real-time, runtime compliance with the contract for transactions throughout the service lifecycle. If a component in a transaction fails to meet its agreed SLA, the entire system (the components in the transaction) may have the ability to (1) understand the impact of the SLA breach, (2) augment other components in the system to restore the overall transaction SLA, and (3) implement corrective steps. In some examples, the cost or penalty of such corrective action may be utilized by the orchestrator 202 when determining workload allocation by using the corrective action as a cost function weight as described above in relation to Figures 3-6.

[0139] Thus, with these variations and service capabilities in mind, edge computing within Edge Cloud 910 can serve and respond to multiple applications in Use Case 1005 (e.g., object tracking, video surveillance, connected vehicles, etc.) in real time or near real time, while also providing the ability to meet the ultra-low latency requirements for these multiple applications. These advantages enable an entirely new class of applications (such as virtual networking functions (VNFs), functions as a service (FaaS), edge as a service (EaaS), and standard processes) that are not available through traditional cloud computing due to latency or other limitations.

[0140] However, the advantages of edge computing come with the following caveats: Devices located at the edge are often resource-constrained, thus putting pressure on the use of edge resources. Typically, this is addressed through pooling memory and storage resources for use by multiple users (tenants) and devices. The edge may have power and cooling constraints, and therefore power usage must be accounted for by the most power-consuming applications. Many of these pooled memory resources are likely to use emerging memory technologies that require more power and greater memory bandwidth, so there may be an inherent power-performance trade-off with these resources. Similarly, since edge locations may be unattended and may even require authorized access (e.g., if housed in a third-party location), improved hardware security and trusted roots of trusted functionality are also required. Such issues are amplified in edge cloud 910 in multi-tenant, multi-owner, or multi-access scenarios where services and applications are requested by a large number of users, especially as network usage fluctuates dynamically and the composition of multiple stakeholders, use cases, and services changes.

[0141] At a more general level, an edge computing system may be described as encompassing any number of deployments in the aforementioned layers (network layers 1000-1040) operating within an edge cloud 910, providing coordination from clients and distributed computing devices. One or more edge gateway nodes, one or more edge aggregation nodes, and one or more core data centers may be distributed across network layers to provide implementations of the edge computing system by, or on behalf of, telecommunications service providers ("telephone companies" or "TSPs"), Internet of Things service providers, cloud service providers (CSPs), enterprise entities, or any number of other entities. Various implementations and configurations of the edge computing system may be provided dynamically, as they are orchestrated to meet service objectives.

[0142] In accordance with the examples provided herein, a client compute node may be embodied as any type of endpoint component, device, appliance, or other capable of communicating as a data producer or consumer. Furthermore, the labels “node” or “device” as used in edge computing systems do not necessarily imply that such a node or device operates in the role of a client or agent / minion / follower; rather, any node or device in an edge computing system refers to an individual entity, node, or subsystem containing discrete or connected hardware or software configurations for facilitating or using the edge cloud.

[0143] Therefore, the edge cloud 910 is formed between network layers 1010-1030 from network components and functional features operated within it by edge gateway nodes, edge aggregation nodes, or other edge compute nodes. Thus, the edge cloud 910 can be embodied as any type of network providing edge computing and / or storage resources, located in close proximity to the radio access network (RAN)-enabled endpoint devices described herein (e.g., mobile computing devices, IoT devices, smart devices, etc.). In other words, the edge cloud 910 may be envisioned as the “edge” connecting the endpoint devices to traditional network access points. Traditional network access points function as entry points to service provider core networks, including mobile carrier networks (e.g., Global Mobile Communications System (GSM) networks, Long-Term Evolution (LTE) networks, 5G / 6G networks, etc.), while also providing storage and / or computing capabilities. Other types and forms of network access (e.g., Wi-Fi, long-range wireless including optical networks, and wired networks) may also be used instead of, or in combination with, such 3GPP carrier networks.

[0144] The network components of Edge Cloud 910 may include servers, multi-tenant servers, appliance computing devices, and / or any other type of computing device. For example, Edge Cloud 910 may include appliance computing devices, which are self-contained electronic devices including a housing, chassis, case, or shell. In some situations, the housing may be sized for portability so that it can be carried and / or transported by a person. An exemplary housing may include materials forming one or more outer surfaces that partially or completely protect the contents of the appliance, and the protection may include weather protection, hazardous environmental protection (e.g., EMI, vibration, extreme temperature), and / or allow submersion in water. An exemplary housing may include power circuits for supplying power for fixed and / or portable implementations, such as AC power inputs, DC power inputs, AC / DC or DC / AC converters, power regulators, transformers, charging circuits, batteries, wired inputs, and / or wireless power inputs. The exemplary housing and / or surface may include or connect to mounting hardware to enable mounting to a building, telecommunications structure (e.g., pole, antenna structure, etc.) and / or rack (e.g., server rack, blade mount, etc.). The exemplary housing and / or surface may support one or more sensors (e.g., temperature sensor, vibration sensor, light sensor, acoustic sensor, capacitive sensor, proximity sensor, etc.). One or more such sensors may be included in the surface of the appliance, carried by the surface, or otherwise embedded in the surface and / or mounted on the surface. The exemplary housing and / or surface may support mechanical connectivity such as propulsion hardware (e.g., wheels, propellers, etc.) and / or articulated hardware (e.g., robotic arm, swivel limb, etc.).In some situations, the sensor may include any type of input device, such as user interface hardware (e.g., buttons, switches, dials, sliders, etc.). In some situations, the exemplary housing may include output devices contained within, carried within, embedded within, and / or attached thereto. Output devices may include displays, touchscreens, lights, LEDs, speakers, I / O ports (e.g., USB), etc. In some situations, the edge device is a device presented in the network for a specific purpose (e.g., traffic signals), but may have processing and / or other capabilities that can be used for other purposes. Such an edge device may be independent of other network devices, may have a housing with a shape factor suitable for its primary purpose, and may also be available for other computing tasks that do not interfere with its primary task. The edge device includes Internet of Things devices. The appliance computing device may include hardware and software components for managing local issues such as device temperature, vibration, resource utilization, updates, power issues, and physical and network security. Exemplary hardware for implementing the appliance computing device is described in relation to Figure 7B. Edge Cloud 910 may also include one or more servers and / or one or more multi-tenant servers. Such servers may include an operating system and a virtual computing environment. The virtual computing environment may include a hypervisor that manages (spawns, deploys, destroys, etc.) one or more virtual machines, one or more containers, etc. Such a virtual computing environment provides an execution environment in which one or more applications and / or other software, code or scripts can run while isolated from one or more other applications, software, code or scripts.

[0145] Figure 11 shows a block diagram of an exemplary environment 1100 in which various client endpoints 1110 (in the form of mobile devices, computers, autonomous vehicles, business computing equipment, and industrial processing equipment) exchange requests and responses with an exemplary edge cloud 910. For example, computers, business computing equipment, and industrial processing equipment can obtain network access via a wired broadband network by exchanging requests and responses 1122 through an on-premise network system 1132. Mobile computing devices can obtain network access via a wireless broadband network by exchanging requests and responses 1124 through a cellular network tower 1134. Autonomous vehicles can obtain network access for requests and responses 1126 via a wireless vehicle network through a street-mounted network system 1136. However, regardless of the type of network access, the TSP can deploy aggregation points 1142, 1144 within the edge cloud 910 to aggregate traffic and requests. Therefore, within the edge cloud 910, the TSP may deploy various computing and storage resources, such as in the edge aggregation node 1140, to provide the requested content. The edge aggregation node 1140 and other systems in the edge cloud 910 are connected to the cloud or data center 1160, which uses the backhaul network 1150 to meet higher latency demands from the cloud / data center for websites, applications, database servers, etc. (Additional or integrated instances of the edge aggregation node 1140 and aggregation points 1142, 1144 may also exist within the edge cloud 910 or other areas of the TSP infrastructure, including those deployed on a single server framework).

[0146] Figure 12 illustrates the deployment and orchestration of a virtual edge configuration across an edge computing system operating across multiple edge nodes and multiple tenants. Specifically, Figure 12 shows the coordination of a first edge node 1222 and a second edge node 1224 in an edge computing system 1200 to meet the demands and responses of various client endpoints 1210 (e.g., smart city / building systems, mobile devices, computing devices, business / logistics systems, industrial systems, etc.) accessing various virtual edge instances. Here, the virtual edge instances provide edge computing capabilities and processing in the edge cloud, with access to the cloud / data center 1240 for higher latency demands such as websites, applications, and database servers. However, the edge cloud enables the coordination of processing across multiple edge nodes for multiple tenants or entities.

[0147] In the example in Figure 12, these virtual edge instances include: a first virtual edge 1232 provided to a first tenant (tenant 1) that provides a first combination of edge storage, compute, and services, and a second virtual edge 1234 that provides a second combination of edge storage, compute, and services. The virtual edge instances 1232, 1234 are distributed among edge nodes 1222, 1224, and may include scenarios where requests and responses are fulfilled from the same or different edge nodes. The configuration of edge nodes 1222, 1224 to operate in a distributed but coordinated manner is based on the edge provisioning function 1250. The functionality of edge nodes 1222, 1224 to provide coordinated operation for applications and services across multiple tenants arises from the orchestration function 1260.

[0148] It should be understood that some devices within 1210 are multi-tenant devices where tenant 1 operates within tenant 1 “slice” and tenant 2 operates within tenant 2 slice (in further examples, additional tenants or subtenants may exist, each tenant being specifically titled down to a particular hardware function and even tied to a specific set of functions transactionally). Trusted multi-tenant devices may further contain tenant-specific cryptographic keys, and the key-slice combination may be considered the “root of trust” or tenant-specific RoT. The RoT may further be dynamically computed and configured using a DICE (Device Identity Composition Engine) architecture, thereby using a single DICE hardware build block (e.g., a field-programmable gate array (FPGA)) to build a trusted compute-based context that is layered for layered configuration of device functions. The RoT may further be used to enable “fan-out” which is useful for the trusted compute context to support multi-tenancy. Within a multi-tenant environment, each edge node 1222, 1224 can act as a security feature enforcement point for local resources allocated to multiple tenants per node. Furthermore, tenant runtimes and application executions (e.g., those in instances 1232, 1234) may act as enforcement points for security features, potentially creating virtual edge abstractions of resources across multiple physical hosting platforms. Finally, the orchestration feature 1260 in the orchestration entity may act as a security feature enforcement point for marshalling resources along tenant boundaries.

[0149] Edge computing nodes can partition resources (memory, CPU, GPU, interrupt controller, I / O controller, memory controller, bus controller, etc.), where each partition may include RoT functionality, and fan-out and layer configurations according to the DICE model may be further applied to the edge nodes. Cloud computing nodes consisting of containers, FaaS engines, servlets, servers, or other computing abstractions may be partitioned according to DICE layer configurations and fan-out structures, each supporting a RoT context. Thus, each device 1210, 1222, and 1240 spanning multiple RoTs can coordinate the establishment of a distributed trusted computing base (DTCB) so that tenant-specific virtual trusted and secure channels linking all elements end-to-end can be established.

[0150] Furthermore, it will be understood that a container may have data or workload-specific keys that protect its contents from the previous edge node. As part of container migration, the pod controller on the source edge node may obtain a migration key from the pod controller on the target edge node, where the migration key is used to wrap the container-specific key. When the container / pod is migrated to the target edge node, the unwrapped key is exposed to the pod controller, which then decrypts the wrapped key. The key can then be used to perform operations on the container-specific data. The migration function may be gated by properly authenticated edge nodes and pod managers (as described above).

[0151] In a further example, edge computing systems are extended to provide orchestration of multiple applications through the use of containers (housed, deployable units of software that provide code and necessary dependencies) in a multi-owner, multi-tenant environment. A multi-tenant orchestrator may be used to perform key management, trust anchor management, and other security functions related to the provisioning and lifecycle of the trusted “slice” concept in Figure 12. For example, an edge computing system may be configured to satisfy requests and responses for various client endpoints from multiple virtual edge instances (and from the cloud or a remote data center). The use of these virtual edge instances can simultaneously support multiple tenants and multiple applications (e.g., augmented reality (AR) / virtual reality (VR), enterprise applications, content delivery, games, compute offloading). Furthermore, multiple types of applications (e.g., normal applications, latency-sensitive applications, latency-critical applications, user-plane applications, networking applications, etc.) may reside within a virtual edge instance. Virtual edge instances may span systems owned by multiple owners in different geographical locations (or each computing system and resource jointly owned or jointly managed by multiple owners).

[0152] For example, edge nodes 1222 and 1224 may each implement the use of containers, such as the use of container "pods" 1226 and 1228 that provide a group of one or more containers. In a scenario where one or more container pods are used, a pod controller or orchestrator is responsible for the local control and orchestration of the containers within the pod. The various edge node resources provided for each edge slice 1232 and 1234 (e.g., storage, computation, and services, depicted as hexagons) are divided according to the needs of each container.

[0153] Regarding the use of container pods, the pod controller oversees the partitioning and allocation of containers and resources. The pod controller receives instructions from an orchestrator (e.g., orchestrator 1260) that instructs the controller on how and for how long it is best to partition physical resources. This is done, for example, by receiving key performance indicator (KPI) targets based on an SLA agreement. The pod controller determines which containers need which resources and for how long to complete the workload and meet the SLA. The pod controller also manages container lifecycle operations, such as creating containers, provisioning resources and applications to containers, coordinating intermediate results between multiple containers working together for a distributed application, and dismantling containers when the workload is complete. Furthermore, the pod controller can play a security role by preventing resource allocation until the correct tenant is authenticated, or by preventing the provisioning of data or workloads to containers until the authentication results are satisfied.

[0154] With regard to the use of container pods, tenant boundaries may still exist, but within the context of each pod in the container. If each tenant-specific pod has its own tenant-specific pod controller, there may be a shared pod controller that consolidates resource allocation requests to avoid typical resource starvation situations. Further controls may be provided to ensure authentication and reliability of pods and pod controllers. For example, orchestrator 1260 may provision an authentication verification policy to a local pod controller that performs authentication verification. If authentication satisfies the policy for the pod controller of the first tenant but not for the pod controller of the second tenant, the second pod may be migrated to a different edge node that satisfies it. Alternatively, the first pod may be allowed to run while a different shared pod controller is installed and invoked before the second pod runs.

[0155] Figure 13 shows additional compute configurations for deploying containers within an edge computing system. As a simplified example, system configurations 1310 and 1320 illustrate scenarios where a pod controller (e.g., container managers 1311, 1321, and 1331) is adapted to launch instances of containerized pods, functions, and functions as services through execution via compute nodes (1315 in configuration 1310), or to separately run containerized virtualized network functions through execution via compute nodes (1323 in configuration 1320). This configuration is adapted for the use of multiple tenants in exemplary system configuration 1330 (using compute node 1336). Here, containerized pods (e.g., pod 1312), functions (e.g., function 1313, VNF 1322, 1336), and function-as-a-service instances (e.g., FaaS instance 1315) are launched within virtual machines specific to each tenant (e.g., VMs 1334, 1335 for tenants 1332, 1333) (apart from the execution of virtualized network functions). This configuration is further adapted for use on compute node 1344 in a system configuration 1340 that provides the execution of containers 1342, 1343, or various functions, applications, and functions, as coordinated by a container-based orchestration system 1341.

[0156] The system configuration shown in Figure 13 provides an architecture that treats VMs, containers, and functions equally with respect to application composition (the resulting application is a combination of these three ingredients). Each ingredient may involve the use of one or more accelerator (FPGA, ASIC) components as a local backend. In this way, applications can be orchestrated and partitioned across multiple edge owners by an orchestrator.

[0157] In the context of Figure 13, the pod controller / container manager, container orchestrator, and individual nodes can provide security enforcement points. However, tenant isolation may be orchestrated. Here, resources allocated to one tenant are different from resources allocated to another tenant, but edge owners cooperate to ensure that resource allocations are not shared across tenant boundaries. Alternatively, resource allocations can be isolated across tenant boundaries, as tenants may grant "use" on a subscription, transaction / contract basis. In these contexts, virtualization, containerization, enclaves, and hardware partitioning schemes may be used by edge owners to enforce tenancy. Other isolation environments may include: bare metal (dedicated) facilities, virtual machines, containers, virtual machines on containers, or a combination thereof.

[0158] In further examples, software-defined or controlled silicon hardware, and other configurable hardware aspects, can integrate edge computing systems with applications, functions, and services. Software-defined silicon can be used to ensure that any resource or hardware component can meet contracts or service level agreements based on its ability to correct itself or parts of its workload (for example, by upgrading, reconfiguring, or providing new functions within the hardware configuration itself).

[0159] It should be understood that the edge computing systems and configurations discussed herein may be applicable to a variety of mobility-related solutions, services, and / or use cases. As an example, Figure 14 shows an exemplary simplified vehicle computing and communications use case involving mobile access to an application in an exemplary edge computing system 1400 implementing an edge cloud such as the edge cloud 910 of Figure 9. In this use case, each client computing node 1410 may be embodied as an in-vehicle computing system (e.g., an in-vehicle navigation and / or information and entertainment system) located within the corresponding vehicle, communicating with an exemplary edge gateway node 1420 while the vehicle is moving along the road. For example, the edge gateway node 1420 may be located in a roadside cabinet or in another enclosure incorporated into a structure having other separate mechanical utilities, which may be located along the road, at a road intersection, or at other locations near the road. As each vehicle moves along the road, the connection between its client compute node 1410 and a particular edge gateway node 1420 can propagate to maintain a consistent connection and context for the exemplary client compute node 1410. Similarly, mobile edge nodes can be aggregated according to high-priority services or (for example, in the case of drones) throughput or latency resolution requirements for the underlying service(s). Each edge gateway node 1420 includes a certain amount of processing and storage capacity, and therefore any processing and / or storage of data for the client compute node 1410 can be performed on one or more of the edge gateway nodes 1420.

[0160] The edge gateway node 1420 can communicate with one or more edge resource nodes 1440, which are exemplary manifestations as compute servers, appliances, or components located in or within a communication base station 1442 (e.g., a base station in a cellular network). As described above, each edge resource node 1440 includes a certain amount of processing and storage capacity, and so any processing and / or storage of data for the client compute node 1410 may be performed on the edge resource node 1440. For example, processing of less urgent or important data may be performed by the edge resource node 1440, while processing of more urgent or important data may be performed by the edge gateway device 1420 (depending on, for example, the capabilities of each component or information in the request indicating urgency or importance). When processing priority changes during processing activity, work may continue on the edge resource node based on data access, data location, or latency. Similarly, configurable system or hardware resources themselves can be activated (for example, through a local orchestrator) to provide additional resources to meet new demands (e.g., to adapt computing resources to workload data).

[0161] The edge resource node 1440 also communicates with the core data center 1450. The core data center may include compute servers, appliances, and / or other components located in a central location (e.g., the central office of a cellular communication network). An exemplary core data center 1450 can provide a gateway to the global network cloud 1460 (e.g., the Internet) for edge cloud 910 operation, which is formed by the edge resource node 1440 and the edge gateway device 1420. Furthermore, in some examples, the core data center 1450 may include a certain amount of processing and storage capacity, and some processing and / or storage of data for client compute devices may be performed on the core data center 1450 (e.g., processing of low urgency or importance, or high complexity).

[0162] An edge gateway node 1420 or edge resource node 1440 may provide access to a stateful application 1432 and a geographically distributed database 1434. While the application 1432 and database 1434 are illustrated as being horizontally distributed in one layer of the edge cloud, it will be understood that the resources, services, or other components of the application may be distributed vertically through the edge cloud (including parts of the application running on client compute nodes 1410 and other parts running on edge gateway nodes 1420 or edge resource nodes 1440, etc.). Furthermore, as previously stated, peer relationships may exist at any level to fulfill the purpose and obligations of the service. In addition, data for a particular client or application may move from edge to edge based on changing conditions (e.g., based on the availability of acceleration resources, following the movement of a vehicle, etc.). For example, predictions can be made based on the "rate of decay" of access to identify the next owner to continue or when data or compute access will no longer be viable. These and other services may be used to complete the tasks required to keep transactions compliant and loss-free.

[0163] In a further scenario, container 1436 (or a pod of containers) may be flexibly migrated from one of the edge nodes 1420 to another edge node (e.g., another edge node 1420, one of the edge resource nodes 1440, 1450, or 1460), so that the container with the application and workload does not need to be rebuilt, recompiled, or reinterpreted for the migration to work. However, in such a scenario, some corrective or "swizzling" transformation operations may be applied. For example, the physical hardware at edge resource node 1440 may differ from the hardware at edge gateway node 1420, so the hardware abstraction layer (HAL) that constitutes the bottom edge of the container is remapped to the physical layer of the target edge node. This may involve some form of late-binding technique, such as the binary conversion of the HAL from the container's native format to the physical hardware format, or it may involve mapping interfaces and operations. The pod controller may be used to drive interface mapping as part of the container lifecycle, including migration to and from different hardware environments.

[0164] The scenarios encompassed in Figure 14 allow for the utilization of various types of mobile edge nodes, such as edge nodes hosted within vehicles (cars / trucks / trams / trains) or other mobile units, as the edge nodes move to other geographical locations along the platform that hosts them. In vehicle-to-vehicle communication, individual vehicles may even act as network edge nodes for other vehicles (e.g., to perform caching, reporting, data aggregation, etc.). Thus, it will be understood that the application components provided at various edge nodes may be distributed in static or mobile settings, including coordination between some functions or operations at individual endpoint devices or edge gateway nodes 1420, some other functions or operations at edge resource nodes 1440, and other functions or operations at the core data center 1450 or global network cloud 1460.

[0165] In a further configuration, edge computing systems can implement FaaS computing capabilities through the use of their respective executable applications and functions. For example, a developer writes function code (e.g., "computer code") representing one or more computer functions, and this function code is uploaded to a FaaS platform provided, for example, by an edge node or data center. A trigger, such as a service use case or edge processing event, initiates the execution of the function code using the FaaS platform.

[0166] In one example of FaaS, containers are used to provide an environment in which functional code (e.g., applications that may be provided by third parties) runs. A container can be any isolated execution entity and may be a process, a Docker or Kubernetes container, a virtual machine, etc. Within an edge computing system, various data centers, edge, and endpoint (including mobile) devices are used to "spin up" functionalities that scale on demand (e.g., activate and / or assign functional actions). The functional code runs on physical infrastructure devices (e.g., edge computing nodes) and the underlying virtualized containers. Finally, the containers are "spinned down" on the infrastructure (e.g., deactivated or deallocated) in response to the completion of their execution.

[0167] Further aspects of FaaS may include enabling the deployment of edge functionality as a service, including support for each functionality that supports Edge-as-a-Service (EaaS). Additional features of FaaS include: a granular billing component where customers (e.g., computer code developers) only pay when their code is executed; common data storage for storing data for reuse by one or more functions; orchestration and management between individual functions; function execution management, parallelism, and integration; management of container and function memory space; coordination of available acceleration resources for functions; and distribution of functions among containers (including "warm" containers that are already deployed and running, as opposed to "cold" containers that require initialization, deployment, or configuration).

[0168] The edge computing system 1400 may include or communicate with an edge provisioning node 1444. The edge provisioning node 1444 can distribute software, such as the exemplary computer-readable instruction 1582 in Figure 15B, to various recipients to implement any of the methods described herein. The exemplary edge provisioning node 1444 may be implemented by any computer server, home server, content distribution network, virtual server, software distribution system, central facility, storage device, storage node, data facility, cloud service, etc., which can store and / or transmit software instructions (e.g., code, scripts, executable binaries, containers, packages, compressed files, and / or derivatives thereof) to and / or other computing devices. Components of the exemplary edge provisioning node 1444 may be located in the cloud, in a local area network, in an edge network, in a wide area network, on the internet, and / or any other location communicatively connected to the recipient(s). The recipient may be a customer, client, stakeholder, or user of the entity that owns and / or operates the edge provisioning node 1444. For example, the entity that owns and / or operates the edge provisioning node 1444 may be the developer, seller, and / or licensor (or its customers and / or consumers) of software instructions such as the exemplary computer-readable instruction 1582 in Figure 15B. The recipient may be a consumer, service provider, user, retailer, OEM, or other person who purchases and / or licenses the software instructions for use and / or resale and / or sublicense.

[0169] In one example, the edge provisioning node 1444 includes one or more servers and one or more storage devices. The storage devices host computer-readable instructions, such as the exemplary computer-readable instruction 1582 in Figure 15B, as described below. Similar to the edge gateway device 1420 described above, the one or more servers of the edge provisioning node 1444 communicate with base station 1442 or other network communication entities. In some examples, the one or more servers respond to requests to send the software instructions to requesters as part of a commercial transaction. Payment for the delivery, sale, and / or licensing of the software instructions may be handled by the one or more servers of the software delivery platform and / or through a third-party payment entity. The servers enable purchasers and / or licensors to download the computer-readable instruction 1582 from the edge provisioning node 1444. For example, software instructions that may correspond to the exemplary computer-readable instruction 1582 in Figure 15B may be downloaded to one or more exemplary processor platforms that execute the computer-readable instruction 1582 in order to implement the methods described herein.

[0170] In some examples, the processor platforms that execute the computer-readable instructions 1582 may be physically located in various geographical locations, legal jurisdictions, etc. In some examples, one or more servers of the edge provisioning node 1444 periodically provide, transmit, and / or enforce updates to the software instructions (e.g., the exemplary computer-readable instructions 1582 in Figure 15B) to ensure that improvements, patches, updates, etc., are distributed and applied to the software instructions implemented in end-user devices. In some examples, different components of the computer-readable instructions 1582 may be distributed from different sources and / or to different processor platforms, for example, different libraries, plugins, components, and other types of compute modules, whether compiled or interpreted, may be distributed from different sources and / or to different processor platforms. For example, a portion of the software instructions (e.g., a script that is not executable in itself) may be distributed from a first source, while the interpreter (which can execute the script) may be distributed from a second source.

[0171] In yet another example, any computing node or device discussed with reference to the edge computing systems and environments of this application may be satisfied based on the components shown in Figures 15A and 15B. Each edge computing node may be embodied as a device, appliance, computer, or other type of “thing” capable of communicating with other edge, networking, or endpoint components. For example, an edge computing device may be embodied as a personal computer, server, smartphone, mobile computing device, smart appliance, in-vehicle computing system (e.g., navigation system), a self-contained device with an outer casing, shell, etc., or other device or system capable of performing the functions described.

[0172] Figure 15A is a block diagram of an exemplary implementation of an exemplary edge compute node 1500, which includes a compute engine (also referred to herein as “Compute Circuits”) 1502, an input / output (I / O) subsystem 1508, data storage 1510, a communication circuit subsystem 1512, and optionally one or more peripheral devices 1514. In other examples, each compute device may include other or additional components, such as those typically found in a computer (e.g., displays, peripherals, etc.). Furthermore, in some examples, one or more exemplary components may be incorporated into another component or otherwise form part of another component. The exemplary edge compute node 1500 in Figure 15 may be deployed in one of the edge computing systems shown in Figures 9–12 and / or Figure 14 to implement any example edge compute node disclosed herein.

[0173] The compute node 1500 may be embodied as any type of engine, device, or collection of devices capable of performing various computing functions. In some examples, the compute node 1500 may be embodied as a single device such as an integrated circuit, embedded system, field-programmable gate array (FPGA), system-on-a-chip (SOC), or other integrated system or device. In an exemplary example, the compute node 1500 includes or is embodied as a processor 1504 and memory 1506. The processor 1504 may be embodied as any type of processor capable of performing the functions described herein (e.g., running applications). For example, the processor 1504 may be embodied as a multicore processor, microcontroller, processing unit, specialized or special-purpose processing unit, or other processor or processing / control circuit.

[0174] In some examples, the processor 1504 may be embodied, include, or be coupled with an FPGA, an application-specific integrated circuit (ASIC), reconfigurable hardware or hardware circuitry, or other specialized hardware to facilitate the performance of the functions described herein. Also in some examples, the processor 1504 may be embodied as a specialized x-processing unit (xPU), also known as a data processing unit (DPU), infrastructure processing unit (IPU), or network processing unit (NPU). Such an xPU may be embodied as a standalone circuit or circuit package, integrated within a SOC, or integrated with network circuitry (e.g., in a SmartNIC), acceleration circuitry, memory, or AI hardware (e.g., a GPU or programmed FPGA). Such an xPU may be designed to process one or more data streams outside of the CPU or general-purpose processing hardware and to receive programming to perform specific tasks and actions for those data streams (e.g., hosting microservices, performing service management or orchestration, organizing or managing server or data center hardware, managing a service mesh, or collecting and distributing telemetry). However, it will be understood that the xPUs, SOCs, CPUs, and other variations of the processor 704 can work together to perform many types of operations and instructions on their behalf within the compute node 1500.

[0175] The main memory 1506 may be embodied as any type of volatile (e.g., dynamic random access memory (DRAM)) or non-volatile memory or data storage device capable of performing the functions described herein. Volatile memory may be a storage medium that requires power to maintain the state of data stored in the medium. Non-limiting examples of volatile memory may include various types of random access memory (RAM), such as DRAM or static random access memory (SRAM). One particular type of DRAM that may be used in a memory module is synchronous dynamic random access memory (SDRAM).

[0176] In one example, the memory device is a block-addressable memory device, such as one based on NAND or NOR technology. The memory device may also include a three-dimensional crosspoint memory device (e.g., Intel® 3D XPoint® memory) or other byte-addressable write-in-place non-volatile memory devices. The memory device may refer to the die itself and / or a packaged memory product. In some examples, 3D crosspoint memory (e.g., Intel® 3D XPoint® memory) may include a transistorless stackable crosspoint architecture in which memory cells are located at the intersections of word lines and bit lines, are individually addressable, and bit storage is based on changes in bulk resistance. In some examples, all or part of the main memory 1506 may be integrated into the processor 1504. The main memory 1506 can store various software and data used during operation, such as one or more applications, data manipulated by the applications, libraries, and drivers.

[0177] The computing circuit 1502 is communicatively coupled to other components of the computing node 1500 via the I / O subsystem 1508, which may be embodied as circuits and / or components to facilitate input / output operations with the computing circuit 1502 (e.g., the processor 1504 and / or main memory 1506) or other components of the computing circuit 1502. For example, the I / O subsystem 1508 may be embodied as a memory controller hub, an input / output control hub, an integrated sensor hub, a firmware device, communication links (e.g., point-to-point links, bus links, wires, cables, optical guides, printed circuit board traces, etc.), and / or other components and subsystems to facilitate input / output operations, or may otherwise include them. In some examples, the I / O subsystem 1508 may form part of a system-on-a-chip (SoC) and be integrated into the computing circuit 1502 together with the processor 1504, the main memory 1506, and one or more of the other components of the computing circuit 1502.

[0178] The one or more exemplary data storage devices 1510 may be embodied as any type of device configured for short-term or long-term storage of data, such as memory devices and circuits, memory cards, hard disk drives, solid-state drives, or other data storage devices. Each data storage device 1510 may include a system partition for storing firmware code and data for the data storage device 1510. Each data storage device 1510 may also include one or more operating system partitions for storing executable programs and data files for the operating system, for example, depending on the type of compute node 1500.

[0179] The communication circuit 1512 may be embodied as any communication circuit, device, or set thereof that can enable communication over a network between the computing circuit 1502 and another computing device (e.g., an edge gateway of an implemented edge computing system). The communication circuit 1512 may be configured to use one or more communication technologies (e.g., wired or wireless) and associated protocols (e.g., cellular networking protocols such as 3GPP 4G or 5G standards, wireless local area network protocols such as IEEE 802.11 / Wi-Fi®, wireless wide area network protocols, Ethernet, Bluetooth®, Bluetooth Low Energy, IoT protocols such as IEEE 802.15.4 or ZigBee®, low-power wide area network (LPWAN) or low-power wide area (LPWA) protocols, etc.) to carry out such communication.

[0180] An exemplary communication circuit 1512 includes a network interface controller (NIC) 1520, which may also be called a host fabric interface (HFI). The NIC 1520 may be embodied as one or more add-in boards, daughter cards, network interface cards, controller chips, chipsets, or other devices that can be used by the compute node 1500 to connect to other computing devices (e.g., edge gateway nodes). In some examples, the NIC 1520 may be embodied as part of a system-on-a-chip (SoC) including one or more processors, or it may be contained on a multi-chip package also including one or more processors. In some examples, the NIC 1520 may include a local processor (not shown) and / or local memory (not shown), both local to the NIC 1520. In such examples, the local processor of the NIC 1520 may be capable of performing one or more functions of the compute circuit 1502 described herein. Additionally or alternatively, in such examples, the local memory of the NIC 1520 may be integrated into one or more components of the client compute node at the board level, socket level, chip level, and / or other levels.

[0181] Furthermore, in some examples, each compute node 1500 may include one or more peripheral devices 1514. Such peripheral devices 1514 may include any type of peripheral device found in a compute device or server, such as audio input devices, displays, other input / output devices, interface devices, and / or other peripheral devices, depending on the particular type of compute node 1500. In further examples, the compute node 1500 may be embodied by each edge compute node (which may be a client, gateway, or aggregation node) in an edge computing system or a similar form of appliance, computer, subsystem, circuit, or other component.

[0182] In a more detailed example, Figure 15B shows a block diagram of an exemplary edge computing node 1550 configured to execute the instructions in Figures 3–6 in order to implement the techniques described herein (e.g., operations, processes, methods, and methodologies), such as the edge platform 200 in Figure 2. This edge computing node 1550 provides a more detailed diagram of each component of node 1500 when implemented as or as part of a computing device (e.g., a mobile device, base station, server, gateway, etc.). The edge computing node 1550 may include any combination of the hardware or logic components referenced herein, and may include or be coupled with any device usable with an edge communication network or a combination of such networks. These components may be implemented as an adapted IC, part thereof, discrete electronic devices, or other modules, instruction sets, programmable logic or algorithms, hardware, hardware accelerators, software, firmware, or a combination thereof, or as components otherwise incorporated into the chassis of a larger system. For example, an edge computing node 1550 could be, for instance, a server, personal computer, workstation, self-learning machine (e.g., neural network), mobile device (e.g., mobile phone, smartphone, tablet like iPad®), personal digital assistant (PDA), internet appliance, DVD player, CD player, digital video recorder, Blu-ray player, game console, personal video recorder, set-top box, headset or other wearable device, Internet of Things (IoT) device, or any other type of computing device.

[0183] The edge computing device 1550 may include processing circuits in the form of a processor 1552, which may be a microprocessor, multicore processor, multithreaded processor, ultra-low voltage processor, embedded processor, xPU / DPU / IPU / NPU, special-purpose processing unit, specialized processing unit, or other known processing element. The processor 1552 may be part of a system-on-a-chip (SoC) in which the processor 1552 and other components are formed as a single integrated circuit or a single package, for example, on an Edison® or Galileo® SoC board from Intel Corporation in Santa Clara, California, USA. As an example, the processor 1552 may include an Intel® Architecture Core® based CPU processor such as a Quark®, Atom®, i3, i5, i7, i9, or MCU-class processor, or another such processor available from Intel®. However, many other processors may be used. For example, these may be available from Advanced Micro Devices, Inc. (AMD®) in Sunnyvale, California, USA; MIPS®-based designs from MIPS Technologies, Inc. in Sunnyvale, California, USA; ARM®-based designs licensed from ARM Holdings, Ltd. or from their customers, or their licensees or adopters. The processors may include units such as A5-A13 processors from Apple® Inc., Snapdragon® processors from Qualcomm® Technologies, Inc., or OMAP® processors from Texas Instruments, Inc. The processor 1552 and associated circuitry may be supplied in single-socket form factor, multi-socket form factor, or a variety of other formats, including limited hardware configurations or configurations containing fewer elements than all of those shown in Figure 15.In this example, the processor 1552 implements an exemplary orchestrator 202, an exemplary function controller 204, an exemplary telemetry controller 206, an exemplary EP database 208, an exemplary resource 210, and / or, more generally, an exemplary edge platform 200.

[0184] The processor 1552 may communicate with system memory 1554 through an interconnect 1556 (e.g., a bus). Any number of memory devices may be used to provide a given amount of system memory. For example, the memory may be random access memory based on Joint Electron Devices Engineering Council (JEDEC) designs such as DDR or mobile DDR standards (e.g., LPDDR, LPDDR2, LPDDR3, or LPDDR4). In specific examples, the memory component may conform to DRAM standards published by JEDEC, such as JESD79F for DDR SDRAM, JESD79-2F for DDR2 SDRAM, JESD79-3F for DDR3 SDRAM, JESD79-4A for DDR4 SDRAM, JESD209 for low-power DDR (LPDDR), JESD209-2 for LPDDR2, JESD209-3 for LPDDR3, and JESD209-4 for LPDDR4. Such standards (and similar standards) may be referred to as DDR-based standards, and the communication interface of a storage device implementing such a standard may be referred to as a DDR-based interface. In various implementations, individual memory devices may be any number of different package types, such as single-die packages (SDP), dual-die packages (DDP), or quad-die packages (Q17P). In some examples, these devices may be soldered directly onto the motherboard to provide a lower-profile solution, while in other examples, these devices consist of one or more memory modules connected to the motherboard by a given connector. Any number of other memory implementations may be used, such as other types of memory modules, including but not limited to microDIMMs or miniDIMMs, and different variants of dual in-line memory modules (DIMMs).

[0185] To provide persistent storage of information such as data, applications, and operating systems, the storage 1558 may also be coupled to the processor 1552 via the interconnect 1556. In one example, the storage 1558 may be implemented via a solid-state disk drive (SSDD). Other devices that may be used for the storage 1558 include flash memory cards such as SD cards, microSD cards, and xD-Picture Cards, and USB flash drives. For example, the memory device may be, or may include, a memory device using chalcogenide glass, a multi-threshold level NAND flash memory, a NOR flash memory, a single or multi-level phase-change memory (PCM), a resistive memory, a nanowire memory, a ferroelectric transistor random-access memory (FeTRAM), an antiferroelectric memory, a magnetoresistive random-access memory (MRAM) incorporating memristor technology, a resistive memory including metal oxide-based, oxygen-vacancy-based, and conductive bridge-type random-access memory (CB-RAM), or a spin-transfer torque (STT)-MRAM, a spintronic magnetic junction memory-based device, a magnetic tunnel junction (MTJ)-based device, a DW (domain wall) and SOT (spin-orbit transition)-based device, a thyristor-based memory device, or any combination thereof, or other memory.

[0186] In low-power implementations, memory 1558 may be an on-die memory or register associated with processor 1552. However, in some examples, memory 1558 may be implemented using a microhard disk drive (HAD). Furthermore, for memory 1558, several new technologies may be used in addition to or instead of the techniques described above. For example, among others, resistive random-access memory, phase-change memory, holographic memory, or chemical memory. 8922.

[0187] Components can communicate through interconnect 1556. Interconnect 1556 may include any number of technologies, including Industrial Standard Architecture (ISA), Extended ISA (EISA), Peripheral Component Interconnect (PCI), Peripheral Component Interconnect Extension (PCIx), PCI Express (PCIe), or several other technologies. Interconnect 1556 may be a proprietary bus used in, for example, SoC-based systems. Other bus systems such as I2C interfaces, SPI interfaces, point-to-point interfaces, and power buses may be included, among others.

[0188] The interconnect 1556 may couple the processor 1552 to the transceiver 1566 for communication with the connected edge device 1562. The transceiver 1566 may use any number of frequencies and protocols, such as 2.4 gigahertz (GHz) transmission under the IEEE 802.15.4 standard, using, in particular, the Bluetooth® Low Energy (BLE) standard or the ZigBee® standard as defined by the Bluetooth® Special Interest Group. Any number of radios configured for a particular wireless communication protocol may be used to connect to the connected edge device 1562. For example, a wireless local area network (WLAN) unit may be used to implement Wi-Fi® communication in accordance with the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard. Furthermore, wireless wide-area communication, such as cellular or other wireless wide-area protocols, may be performed via a wireless wide-area network (WWAN) unit.

[0189] A wireless network transceiver 1566 (or multiple transceivers) may communicate using multiple standards or radios for communication at different ranges. For example, an edge computing node 1550 may communicate with nearby devices, for example within about 10 meters, using a BLE-based local transceiver or other low-power radio to conserve power. A connected edge device 1562 at a greater distance, for example within about 50 meters, may be reached via a ZigBee® or other intermediate-power radio. Both communication techniques may be performed through a single radio at different power levels, or through separate transceivers, for example, a local transceiver using BLE and separate mesh transceivers using ZigBee®.

[0190] A wireless network transceiver 1566 (for example, a wireless transceiver) may be included to communicate with devices or services within the edge cloud 1590 via local or wide-area network protocols. The wireless network transceiver 1566 may, among other things, be an LPWA transceiver conforming to the IEEE 802.15.4 or IEEE 802.15.4g standard. The edge computing node 1550 can communicate over a wide area using LoraWAN® (Long Range Wide Area Network), developed by Semtech and the LoRa Alliance. The techniques described herein are not limited to these techniques and may be used in conjunction with any number of other cloud transceivers implementing long-range, low-bandwidth communication and other techniques, such as Sigfox. Furthermore, other communication techniques, such as time-slotted channel hopping as described in the IEEE 802.15.4e specification, may be used.

[0191] In addition to the systems referred to for the wireless network transceiver 1566 described herein, any number of other wireless communications and protocols may be used. For example, transceiver 1566 may include a cellular transceiver that uses spread spectrum (SPA / SAS) communication to implement high-speed communication. Furthermore, any number of other protocols may be used, such as Wi-Fi® networks for medium-speed communication or for providing network communications. Transceiver 1566 may include radios compatible with any number of 3GPP (Third Generation Partnership Project) specifications, such as Long-Term Evolution (LTE) and fifth-generation (5G) communication systems, which are described in more detail at the end of this disclosure. A network interface controller (NIC) 1568 may be included to provide wired communication to nodes or other devices of the edge cloud 1590, for example, connected edge devices 1562 (for example, operating in a mesh). Wired communication may provide an Ethernet connection or be based on other types of networks, such as, among others, Controller Area Network (CAN), Local Interconnection Network (LIN), DeviceNet, ControlNet, Data Highway+, PROFIBUS, or PROFINET. Additional NICs 1568 may be included to enable connectivity to a second network, for example, the first NIC 1568 providing communication to the cloud via Ethernet and the second NIC 1568 providing communication to other devices via another type of network.

[0192] Because the types of communication applicable from the device to other components or networks are diverse, the applicable communication circuits used by the device may include, or be embodied by, one or more of components 1564, 1566, 1568, or 1570. Thus, in various examples, applicable means for communication (e.g., receiving, transmitting, etc.) may be embodied by such communication circuits.

[0193] The edge computing node 1550 may include or be coupled to an accelerator 1564, which may be embodied by one or more AI accelerators, neural computing sticks, neural morphological hardware, FPGAs, GPU configurations, xPU / DPU / IPU / NPU configurations, one or more SoCs, one or more CPUs, one or more digital signal processors, dedicated ASICs, or other forms of specialized processors or circuits designed to accomplish one or more specialized tasks. These tasks may include AI processing (including machine learning, training, inference, and classification operations), visual data processing, network data processing, object detection, rule analysis, etc. These tasks may also include specific edge computing tasks for service management and service operation as discussed elsewhere in this paper.

[0194] The interconnect 1556 may connect the processor 1552 to a sensor hub or external interface 1570 used to connect additional devices or subsystems. The devices may include sensors 1572 such as accelerometers, level sensors, flow sensors, optical sensors, camera sensors, temperature sensors, global navigation system (e.g., GPS) sensors, pressure sensors, and barometric pressure sensors. The hub or interface 1570 may further be used to connect the edge computing node 1550 to actuators 1574 such as power switches, valve actuators, audible sound generators, and visual warning devices.

[0195] In some optional examples, various input / output (I / O) devices may reside within or be connected to the edge computing node 1550. For example, a display device or other output device 1584 may be included to show information such as sensor readings or actuator positions. An input device 1586, such as a touchscreen or keypad, may be included to accept input. Output devices 1584 may include several forms of audio or visual displays, including simple visual outputs such as two-state indicators (e.g., LEDs) and multi-character visual outputs, or more complex outputs such as display screens (e.g., LCD screens), which have outputs such as characters, graphics, and multimedia objects generated or produced from the operation of the edge computing node 1550. In the context of this system, the display or console hardware may be used to provide outputs to the edge computing system, receive inputs; manage components or services of the edge computing system; identify the state of edge computing components or services; or perform several other arbitrary management or admin functions or service use cases.

[0196] Battery 1576 may power the edge computing node 1550, but in examples where the edge computing node 1550 is mounted in a fixed location, it may have a power source coupled to the electrical grid, or the battery may be used as a backup or for temporary capacity. Battery 1576 may be a lithium-ion battery, or a metal-air battery such as a zinc-air battery, aluminum-air battery, or lithium-air battery.

[0197] The battery monitor / charger 1578 may be included in the edge compute node 1550, and if included, may track the charge state (SoCh) of battery 1576. The battery monitor / charger 1578 may also be used to monitor other parameters of battery 1576 to provide fault prediction, such as the state of health (SoH) and state of function (SoF) of battery 1576. The battery monitor / charger 1578 may include a battery monitoring integrated circuit such as the LTC4020 or LTC2990 (Linear Technologies), ADT7488A (ON Semiconductor, Phoenix, Arizona), or an IC from the UCD90xxx family (Texas Instruments, Dallas, Texas). The battery monitor / charger 1578 may communicate information about battery 1576 to processor 1552 via interconnect 1556. The battery monitor / charger 1578 may also include an analog-to-digital converter (ADC) that allows the processor 1552 to directly monitor the voltage of the battery 1576 or the current from the battery 1576. Battery parameters may be used to determine the operations that the edge computing node 1550 can perform, such as the transmit frequency, mesh network operation, and sensing frequency.

[0198] To charge battery 1576, a power block 1580, or another grid-coupled power source, may be coupled to the battery monitor / charger 1578. In some examples, the power block 1580 may be replaced by a wireless power receiver to obtain power wirelessly, for example, through a loop antenna in an edge computing node 1550. A wireless battery charging circuit, such as the LTC4020 chip from Linear Technologies of Milpitas, California, may be included in the battery monitor / charger 1578. The specific charging circuit may be selected based on the size of battery 1576 and therefore the current required. Charging may be performed using the Airfuel standard published by the Airfuel Alliance, the Qi wireless charging standard published by the Wireless Power Consortium, or the Rezence charging standard published by the Alliance for Wireless Power, etc.

[0199] Memory 1558 may contain instructions 1582 in the form of software, firmware, or hardware commands for implementing the techniques described herein. Such instructions 1582 are shown as code blocks contained in memory 1554 and memory 1558, but it will be understood that any of the code blocks can be replaced, for example, with fixed-connection circuits incorporated in an application-specific integrated circuit (ASIC).

[0200] For example, an instruction 1582 provided via memory 1554, storage 1558, or processor 1552 may be embodied as a non-temporary machine-readable medium 1560 containing code that instructs the processor 1552 to perform electronic operations within the edge computing node 1550. The processor 1552 can access the non-temporary machine-readable medium 1560 through interconnect 1556. For example, the non-temporary machine-readable medium 1560 may be embodied by the devices described for storage 1558, or it may include a specific storage unit such as an optical disc, a flash drive, or any other hardware device. The non-temporary machine-readable medium 1560 may contain instructions that instruct the processor 1552 to perform a specific sequence or flow of actions, as described with respect to the flowcharts and block diagrams of operations and functions described above. As used herein, the terms “machine-readable medium” and “computer-readable medium” are interchangeable.

[0201] In further examples, machine-readable media also include any tangible medium that can store, encode, or carry instructions for machine execution, causing the machine to execute one or more of the methods of the Disclosure, or that can store, encode, or carry data structures utilized by or related to such instructions. Thus, “machine-readable media” may include, but is not limited to, solid memory, and optical and magnetic media. Specific examples of machine-readable media include semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)), flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and non-volatile memory including, but not limited to, CD-ROM and DVD-ROM disks. Instructions embodied by machine-readable media may be transmitted or received over a communication network using a transmission medium via a network interface device utilizing any of several transfer protocols (e.g., HTTP).

[0202] The machine-readable medium may be provided by a storage device or other device capable of hosting data in a non-temporary format. For example, information stored on or otherwise provided on the machine-readable medium may represent instructions, such as the instructions themselves or a format from which instructions can be derived. This format from which instructions can be derived may include source code, encoded instructions (e.g., in a compressed or encrypted form), packaged instructions (e.g., divided into multiple packages), etc. Information representing instructions on the machine-readable medium may be processed by a processing circuit to become instructions for implementing any of the operations discussed herein. For example, deriving instructions from information (e.g., processing by a processing circuit) may include: compiling (e.g., from source code, object code, etc.), interpreting, loading, organizing (e.g., dynamically or statically linking), encoding, decoding, encrypting, decrypting, packaging, unpackaging, or otherwise manipulating the information into the instructions.

[0203] In one example, instruction derivation may involve assembling, compiling, or interpreting information (e.g., by a processing circuit) to generate the instruction from some intermediate or pre-processed format provided by a machine-readable medium. If the information is provided in multiple parts, it may be combined, depackaged, and modified to create the instruction. For example, the information may be in multiple compressed source code packages (or object code, or binary executable code, etc.) on one or more remote servers. The source code packages may be encrypted in transit over a network, decrypted, decompressed, assembled (e.g., linked) as needed, compiled or interpreted on a local machine (e.g., into a library, a standalone executable program, etc.), and executed by that local machine.

[0204] The machine-executable instructions 1582 may be stored in the mass storage device 728, in the volatile memory 714, in the non-volatile memory 716, and / or on a removable, non-temporary, computer-readable storage medium such as a CD or DVD.

[0205] From the above, it will be understood that the exemplary methods, apparatus, and products are disclosed for workload placement in edge environments. The disclosed methods, apparatus, and products improve the efficiency of using computing equipment by adjusting the computing resources spent on orchestrating the requests to perform the workload at the workload, based on the available power and / or thermal levels of the edge platform. Thus, the disclosed methods, apparatus, and products are directed toward one or more improvements in the capabilities of computers.

[0206] Further examples and combinations include: Example 1 is an apparatus: An orchestrator that receives requests to execute workloads from edge platforms within an edge environment; The requirements are analyzed to determine the operational parameters for the workload from the edge platform; Based on the aforementioned operating parameters, the candidate edge layer and edge platform placement are analyzed. The system includes a function controller, and the orchestrator determines candidate edge layer and edge platform arrangements for the workload based on candidate edge layer and edge platform arrangements that satisfy the operating parameters. Includes the device.

[0207] Example 2 includes the apparatus described in Example 1, wherein the function controller derives a subset of factors required for the operating parameters when determining the operating parameters.

[0208] Example 3 includes the apparatus described in Example 1 or 2, wherein a subset of the factors includes the service level agreement (SLA) attribute of the edge platform.

[0209] Example 4 includes the apparatus described in any one of Examples 1 to 3, wherein the subset of factors includes at least one of the following: the level of security, the level of accuracy, the dependency of the edge platform to other microservices, the location of the dependency in the edge environment, or usage information of the edge platform.

[0210] Example 5 includes the apparatus described in any one of Examples 1 to 4, wherein the function controller estimates the resource requirements for the edge platform, and the resource requirements include power measurement, thermal measurement, latency measurement, and bandwidth measurement for the workload based on a subset of the factors.

[0211] Example 6 includes an apparatus of the apparatus described in any one of Examples 1 to 5, wherein the function controller updates the resource requirements based on updated operating parameters.

[0212] The orchestrator includes the apparatus described in any one of Examples 1 to 6, which receives telemetry data from each of the candidate edge layers and edge platform configurations.

[0213] Example 8 includes the apparatus described in any one of Examples 1 to 7, wherein the function controller determines the resource availability of each of the candidate edge layer and edge platform configurations, and the resource availability corresponds to power availability, thermal availability, latency availability, and bandwidth availability in each of the candidate edge layer and edge platform configurations, based on the telemetry data.

[0214] Example 9 is, The aforementioned orchestrator is: A step of determining the operational parameter ratio for each of the candidate edge layers and edge platform configurations, wherein the operational parameter ratio corresponds to the ratio of resource availability for a given edge layer and edge platform configuration that satisfies the resource requirements of the edge platform; The step of selecting an edge layer and edge platform arrangement having the optimal ratio of operating parameters as the candidate edge layer and edge platform arrangement; The steps include performing the aforementioned workload in the candidate edge layer and edge platform placement. The apparatus includes the one described in any one of Examples 1 to 8.

[0215] Example 10 includes the apparatus described in any one of Examples 1 to 9, wherein the orchestrator adjusts resource requirements and cost function weights in response to the operating parameter ratios for each of the candidate edge layers and edge platform placements not meeting a threshold.

[0216] Example 11 is an apparatus: An orchestration means for receiving requests to execute workloads from edge platforms within an edge environment; The requirements are analyzed to determine the operational parameters for the workload from the edge platform; Based on the aforementioned operating parameters, the candidate edge layer and edge platform placement are analyzed. The system includes a function control means, and the orchestration means determines the candidate edge layer and edge platform arrangement for the workload based on the candidate edge layer and edge platform arrangement that satisfies the operating parameters. Includes the device.

[0217] Example 12 includes the apparatus described in Example 11, wherein the function control means derives a subset of factors required for the operation parameters when determining the operation parameters.

[0218] Example 13 includes the apparatus described in Example 11 or 12, wherein a subset of the factors includes the service level agreement (SLA) attribute of the edge platform.

[0219] Example 14 includes the apparatus described in any one of Examples 11 to 13, wherein a subset of the factors includes at least one of the following: the level of security, the level of accuracy, the dependency of the edge platform to other microservices, the location of the dependency in the edge environment, or usage information of the edge platform.

[0220] Example 15 includes the apparatus described in any one of Examples 11 to 14, wherein the function control means estimates resource requirements for the edge platform, and the resource requirements include power measurement, thermal measurement, latency measurement, and bandwidth measurement for the workload based on a subset of the factors.

[0221] Example 16 includes the apparatus described in any one of Examples 11 to 15, wherein the function control means updates the resource requirements based on updated operating parameters.

[0222] Example 17 includes the apparatus described in any one of Examples 11 to 16, wherein the orchestration means receives telemetry data from each of the candidate edge layers and edge platform arrangements.

[0223] Example 18 includes the apparatus described in any one of Examples 11 to 17, wherein the function control means determines the resource availability of each of the candidate edge layer and edge platform configurations, and the resource availability corresponds to power availability, thermal availability, latency availability, and bandwidth availability in each of the candidate edge layer and edge platform configurations, based on the telemetry data.

[0224] Example 19 is that the orchestration means is: A step of determining the operational parameter ratio for each of the candidate edge layers and edge platform configurations, wherein the operational parameter ratio corresponds to the ratio of resource availability for a given edge layer and edge platform configuration that satisfies the resource requirements of the edge platform; The step of selecting an edge layer and edge platform arrangement having the optimal ratio of operating parameters as the candidate edge layer and edge platform arrangement; The steps include performing the aforementioned workload in the candidate edge layer and edge platform placement. This includes the apparatus described in any one of Examples 11 to 18.

[0225] Example 20 includes the apparatus described in any one of Examples 1 to 19, wherein the orchestration means adjusts resource requirements and cost function weights in response to the operating parameter ratios for each of the candidate edge layers and edge platform arrangements not meeting a threshold.

[0226] Example 21 is a non-temporary computer-readable medium having data, wherein the data can be configured into executable instructions, and when configured and executed, it can be performed by at least one processor through at least the following steps, namely: The stage of accessing requests to execute workloads from edge platforms within the edge environment; The steps include: analyzing the request in order to determine the operational parameters for the workload from the edge platform; The steps include: analyzing candidate edge layer and edge platform placement based on the aforementioned operating parameters; The process involves performing the step of determining candidate edge layer and edge platform arrangements for the workload based on candidate edge layer and edge platform arrangements that satisfy the aforementioned operating parameters. Includes non-temporary computer-readable media.

[0227] Embodiment 22 includes a non-temporary computer-readable medium as in Embodiment 21, wherein the instruction, when configured and executed, causes the at least one processor to derive a subset of factors required for the operating parameters when determining the operating parameters.

[0228] Example 23 includes a non-transient computer-readable medium as described in Example 21 or 22, wherein a subset of the factors includes the Service Level Agreement (SLA) attribute of the edge platform.

[0229] Example 24 includes a non-temporary computer-readable medium as described in any one of Examples 21 to 23, wherein a subset of the factors includes at least one of the following: the level of security, the level of accuracy, the dependency of the edge platform to other microservices, the location of the dependency in the edge environment, or usage information of the edge platform.

[0230] Example 25 provides a non-transient computer-readable medium as described in any one of Examples 21 to 24, wherein the instruction, when configured and executed, causes the at least one processor to estimate resource requirements for the edge platform, the resource requirements including power measurements, thermal measurements, latency measurements, and bandwidth measurements for the workload based on a subset of the factors.

[0231] Embodiment 26 includes a non-temporary computer-readable medium as described in any one of Embodiments 21 to 25, wherein the instruction, when configured and executed, causes the at least one processor to update the resource requirements based on updated operating parameters.

[0232] Embodiment 27 includes a non-transient computer-readable medium as described in any one of Embodiments 21 to 26, which, when configured and executed, causes the at least one processor to receive telemetry data from each of the candidate edge layers and edge platform configurations.

[0233] Embodiment 28 provides an instruction that, when configured and executed, causes the at least one processor to determine the resource availability of each of the candidate edge layer and edge platform configurations, the resource availability including a non-transient computer-readable medium as described in any one of Embodiments 1 to 27, corresponding to power availability, thermal availability, latency availability, and bandwidth availability at each of the candidate edge layer and edge platform configurations, based on the telemetry data.

[0234] Embodiment 29 describes how, when the instruction is configured and executed, it is sent to at least one processor: A step of determining the operational parameter ratio for each of the candidate edge layers and edge platform configurations, wherein the operational parameter ratio corresponds to the ratio of resource availability for a given edge layer and edge platform configuration that satisfies the resource requirements of the edge platform; The step of selecting an edge layer and edge platform arrangement having the optimal ratio of operating parameters as the candidate edge layer and edge platform arrangement; The steps involve performing the aforementioned workload in the candidate edge layer and edge platform configuration. This includes a non-temporary computer-readable medium as described in any one of Examples 1 to 28.

[0235] Embodiment 30 includes a non-temporary computer-readable medium as described in any one of Embodiments 21 to 29, wherein the instruction, when configured and executed, causes the at least one processor to adjust resource requirements and cost function weights in response to the operating parameter ratios for each of the candidate edge layers and edge platform placements not meeting a threshold.

[0236] Example 31 is a method: The stage of receiving requests to execute workloads from edge platforms within the edge environment; The steps include: analyzing the request in order to determine the operational parameters for the workload from the edge platform; The steps include: analyzing candidate edge layer and edge platform placement based on the aforementioned operating parameters; The step includes determining candidate edge layer and edge platform arrangements for the workload based on candidate edge layer and edge platform arrangements that satisfy the aforementioned operating parameters, Includes methods.

[0237] Example 32 includes the method of Example 31, wherein determining the operating parameters involves deriving a subset of the factors required for the operating parameters.

[0238] Example 33 includes the method of Example 31 or 32, wherein a subset of the factors includes the Service Level Agreement (SLA) attribute of the edge platform.

[0239] Example 34 includes the method of any one of Examples 31 to 33, wherein a subset of the factors includes at least one of the following: the level of security, the level of accuracy, the dependency of the edge platform to other microservices, the location of the dependency in the edge environment, or usage information of the edge platform.

[0240] Example 35 includes determining the operating parameters and estimating resource requirements for the edge platform, wherein the resource requirements include the method of any one of Examples 31 to 34, including power measurement, thermal measurement, latency measurement, and bandwidth measurement for the workload, based on a subset of the factors.

[0241] Example 36 includes the method of any one of Examples 31 to 35, further comprising the step of updating the resource requirements based on the updated operating parameters.

[0242] Example 37 further includes the method of any one of Examples 1 to 36, comprising the step of receiving telemetry data from each of the candidate edge layers and edge platform configurations.

[0243] Example 38 includes the method of any one of Examples 31 to 37, wherein the analysis of the candidate edge layer and edge platform configurations includes determining the resource availability of each of the candidate edge layer and edge platform configurations, the resource availability of which corresponds to power availability, thermal availability, latency availability, and bandwidth availability for each of the candidate edge layer and edge platform configurations, based on the telemetry data.

[0244] Example 39 describes the determination of the candidate edge layer and edge platform arrangement for the edge platform as follows: A step of determining the operational parameter ratio for each of the candidate edge layers and edge platform configurations, wherein the operational parameter ratio corresponds to the ratio of resource availability for a given edge layer and edge platform configuration that satisfies the resource requirements of the edge platform; The step of selecting an edge layer and edge platform arrangement having the optimal ratio of operating parameters as the candidate edge layer and edge platform arrangement; This includes the step of performing the aforementioned workload in the candidate edge layer and edge platform placement, This includes the method described in any one of Examples 31 to 38.

[0245] Example 40 further includes the method of any one of Examples 1 to 39, further comprising adjusting resource requirements and cost function weights in response to the operating parameter ratios for each of the candidate edge layers and edge platform configurations not meeting a threshold.

[0246] Example 41 is a non-temporary computer-readable medium containing instructions for performing any of Examples 31 to 40.

[0247] Example 42 is an edge computing gateway equipped with a processing circuit for performing any of Examples 31 to 40.

[0248] Example 43 is a method: The stage of receiving a request to run a Cloudlet in the edge environment; The steps include: analyzing the request in order to determine the operating parameters for the aforementioned cloudlet; The steps include: analyzing candidate edge layer and edge platform placement based on the aforementioned operating parameters; The step includes determining candidate edge layer and edge platform placements for the cloudlet based on candidate edge layer and edge platform placements that satisfy the aforementioned operating parameters, Includes methods.

[0249] Example 44 includes the method of Example 43, wherein determining the operating parameters involves deriving a subset of the factors required for the operating parameters.

[0250] Example 45 includes the method of Example 43 or 44, wherein a subset of the factors includes the Service Level Agreement (SLA) attribute of the Cloudlet.

[0251] Example 46 includes the method of any one of Examples 43 to 45, wherein a subset of the factors includes at least one of the following: the level of security, the level of accuracy, the Cloudlet's dependency on other microservices, the location of the dependency in the edge environment, or the Cloudlet's usage information.

[0252] Example 47 includes determining the operating parameters and estimating the resource requirements for the cloudlet, wherein the resource requirements include the method of any one of Examples 43 to 46, including power measurement, thermal measurement, latency measurement, and bandwidth measurement for the cloudlet, based on a subset of the factors.

[0253] Example 48 includes the method of any one of Examples 43 to 47, further comprising the step of updating the resource requirements based on the updated operating parameters.

[0254] Example 49 includes the method of any one of Examples 43 to 48, further comprising the step of receiving telemetry data from each of the candidate edge layers and edge platform configurations.

[0255] Example 50 includes the method of any one of Examples 43 to 49, wherein the analysis of the candidate edge layer and edge platform configurations includes determining the resource availability of each of the candidate edge layer and edge platform configurations, the resource availability of which corresponds to power availability, thermal availability, latency availability, and bandwidth availability for each of the candidate edge layer and edge platform configurations, based on the telemetry data.

[0256] Example 51 describes the determination of the candidate edge layer and edge platform placement for the Cloudlet as follows: A step of determining the operational parameter ratio for each of the candidate edge layers and edge platform configurations, wherein the operational parameter ratio corresponds to the ratio of resource availability for a given edge layer and edge platform configuration that satisfies the resource requirements of the Cloudlet; The step of selecting an edge layer and edge platform arrangement having the optimal ratio of operating parameters as the candidate edge layer and edge platform arrangement; This includes the step of implementing the cloudlet in the candidate edge layer and edge platform deployment, This includes the method described in any one of Examples 43 to 50.

[0257] Example 52 includes the method of any one of Examples 43 to 51, further comprising adjusting resource requirements and cost function weights in response to the operating parameter ratios for each of the candidate edge layers and edge platform configurations not meeting a threshold.

[0258] Example 53 is a non-temporary computer-readable medium containing instructions for performing any of Examples 43 to 52.

[0259] Example 54 is an edge computing gateway equipped with a processing circuit for performing any of Examples 43 to 52.

[0260] Example 55 is a method: The stage of receiving a request to run a Cloudlet in the edge environment; The steps include: analyzing the request in order to determine the operating parameters for the aforementioned cloudlet; The steps include: analyzing candidate edge layer arrangements based on the aforementioned operating parameters; The step includes determining a candidate edge layer arrangement for the cloudlet based on candidate edge layer arrangements that satisfy the aforementioned operating parameters, Includes methods.

[0261] Example 56 includes the method of Example 56, wherein determining the operating parameters includes deriving a subset of the factors required for the operating parameters.

[0262] Example 57 includes the method of Example 55 or 56, wherein a subset of the factors includes the Service Level Agreement (SLA) attribute of the Cloudlet.

[0263] Example 58 includes the method of any one of Examples 55 to 57, wherein a subset of the factors includes at least one of the following: the level of security, the level of accuracy, the Cloudlet's dependency on other microservices, the location of the dependency in the edge environment, or the Cloudlet's usage information.

[0264] Example 59 includes determining the operating parameters, estimating resource requirements for the cloudlet, the resource requirements including the method of any one of Examples 55 to 58, which includes power measurement, thermal measurement, latency measurement, and bandwidth measurement for the cloudlet, based on a subset of the factors.

[0265] Example 60 includes the method of any one of Examples 55 to 59, further comprising the step of updating the resource requirements based on the updated operating parameters.

[0266] Example 61 further includes the method of any one of Examples 55 to 60, comprising the step of receiving telemetry data from each of the candidate edge layer arrangements.

[0267] Example 62, wherein the analysis of the candidate edge layer arrangement includes determining the resource availability of each of the candidate edge layer arrangements, the resource availability corresponding to power availability, thermal availability, latency availability, and bandwidth availability in each of the candidate edge layer arrangements based on the telemetry data, and including the method according to any one of Examples 55 to 61.

[0268] Example 63, wherein the determination of the candidate edge layer arrangement for the cloudlet comprises: determining, for each of the candidate edge layer arrangements, an operating parameter ratio corresponding to a ratio of the resource availability of an edge layer arrangement that satisfies the resource requirements of the cloudlet; selecting, as the candidate edge layer arrangement, an edge layer arrangement having an optimal operating parameter ratio; and implementing the cloudlet in the candidate edge layer arrangement, including the method according to any one of Examples 55 to 62.

[0269] Example 64, further including adjusting resource requirements and cost function weights in response to the operating parameter ratio for each of the candidate edge layer arrangements not meeting a threshold, and including the method according to any one of Examples 55 to 63.

[0270] Example 65 is a non - transient computer - readable medium including instructions for executing any one of Examples 55 to 64.

[0271] Example 66 is an edge - computing gateway comprising a processing circuit for executing any one of Examples 55 to 64.

[0272] Certain exemplary methods, apparatuses, and products are disclosed herein, but the scope of this patent is not limited thereto. Conversely, this patent covers all methods, apparatuses, and products that fall within the scope of the claims herein.

[0273] The following claims are incorporated by this reference into this detailed description, and each claim stands alone as a separate embodiment of the present disclosure.

Claims

1. A device for workload allocation in an edge environment, which includes: An orchestrator that receives requests to execute workloads from the requesting edge platform within the edge environment; The request is analyzed to determine the operational parameters for the workload from the requesting edge platform; Based on the aforementioned operating parameters, the candidate edge layer and edge platform placement are analyzed. The system includes a function controller, and the orchestrator determines candidate edge layer and edge platform arrangements for the workload based on candidate edge layer and edge platform arrangements that satisfy the operating parameters. Determining candidate edge layer and edge platform placements for the aforementioned workload involves: A step of determining the operational parameter ratio for each of the candidate edge layers and edge platform configurations, wherein the operational parameter ratio corresponds to the ratio of resource availability of a given edge layer and edge platform configuration that satisfies the resource requirements of the requesting edge platform; This includes the step of selecting an edge layer and edge platform arrangement having the optimal ratio of operating parameters as the candidate edge layer and edge platform arrangement. Device.

2. The apparatus according to claim 1, wherein the function controller derives a subset of factors required for the operation parameters when determining the operation parameters.

3. The apparatus according to claim 2, wherein a subset of the factors includes the service level agreement (SLA) attributes of the requesting edge platform.

4. The apparatus according to claim 2, wherein the subset of the factors includes at least one of the security level and the accuracy level.

5. The apparatus according to any one of claims 2 to 4, wherein the function controller estimates resource requirements for the requesting edge platform, and the resource requirements include power measurement, thermal measurement, latency measurement, and bandwidth measurement for the workload based on a subset of the factors.

6. The apparatus according to claim 5, wherein the function controller updates the resource requirements based on updated operating parameters.

7. A device for workload allocation in an edge environment, which includes: An orchestration means that receives requests to execute workloads from the requesting edge platform within the edge environment; The request is analyzed to determine the operational parameters for the workload from the requesting edge platform; Based on the aforementioned operating parameters, the candidate edge layer and edge platform placement are analyzed. The system includes a function control means, and the orchestration means determines the candidate edge layer and edge platform arrangement for the workload based on the candidate edge layer and edge platform arrangement that satisfies the operating parameters. Determining candidate edge layer and edge platform placements for the aforementioned workload involves: A step of determining the operational parameter ratio for each of the candidate edge layers and edge platform configurations, wherein the operational parameter ratio corresponds to the ratio of resource availability of a given edge layer and edge platform configuration that satisfies the resource requirements of the requesting edge platform; This includes the step of selecting an edge layer and edge platform arrangement having the optimal ratio of operating parameters as the candidate edge layer and edge platform arrangement. Device.

8. The apparatus according to claim 7, wherein the function control means derives a subset of factors required for the operation parameter when determining the operation parameter.

9. The apparatus according to claim 8, wherein a subset of the factors includes the service level agreement (SLA) attribute of the edge platform.

10. The apparatus according to claim 8, wherein the subset of the factors includes at least one of the security level and the accuracy level.

11. The apparatus according to any one of claims 8 to 10, wherein the function control means estimates resource requirements for the requesting edge platform, and the resource requirements include power measurement, thermal measurement, latency measurement, and bandwidth measurement for the workload based on a subset of the factors.

12. The apparatus according to claim 11, wherein the function control means updates the resource requirements based on updated operating parameters.

13. A non-temporary computer-readable medium having data that can be configured into executable instructions, wherein, when configured and executed, the data can be used by at least one processor to perform at least the following steps: The stage of accessing requests to execute workloads from the requesting edge platform within the edge environment; The steps include: analyzing the request in order to determine the operational parameters for the workload from the requesting edge platform; The steps include: analyzing candidate edge layer and edge platform placement based on the aforementioned operating parameters; The process involves performing the step of determining candidate edge layer and edge platform arrangements for the workload based on candidate edge layer and edge platform arrangements that satisfy the aforementioned operating parameters. Determining candidate edge layer and edge platform placements for the aforementioned workload involves: A step of determining the operational parameter ratio for each of the candidate edge layers and edge platform configurations, wherein the operational parameter ratio corresponds to the ratio of resource availability of a given edge layer and edge platform configuration that satisfies the resource requirements of the requesting edge platform; This includes the step of selecting an edge layer and edge platform arrangement having the optimal ratio of operating parameters as the candidate edge layer and edge platform arrangement. A non-temporary computer-readable medium.

14. The non-temporary computer-readable medium according to claim 13, wherein the instruction, when configured and executed, causes the at least one processor to derive a subset of factors required for the operating parameters when determining the operating parameters.

15. A non-transient computer-readable medium according to claim 14, wherein a subset of the factors includes the service level agreement (SLA) attributes of the edge platform.

16. The non-temporary computer-readable medium according to claim 14, wherein a subset of the factors includes at least one of a security level and a precision level.

17. The instruction, when configured and executed, causes the at least one processor to estimate resource requirements for the requesting edge platform, the resource requirements including power measurement, thermal measurement, latency measurement, and bandwidth measurement for the workload based on a subset of the factors, according to any one of claims 14 to 16, a non-transient computer-readable medium.

18. The non-temporary computer-readable medium according to claim 17, wherein the instruction, when configured and executed, causes the at least one processor to update the resource requirements based on the updated operating parameters.

19. A method for workload allocation in an edge environment, which includes: The stage of receiving a request to execute a workload from the requesting edge platform within the edge environment; The steps include: analyzing the request in order to determine the operational parameters for the workload from the requesting edge platform; The steps include: analyzing candidate edge layer and edge platform placement based on the aforementioned operating parameters; The process includes the step of determining candidate edge layer and edge platform arrangements for the workload based on candidate edge layer and edge platform arrangements that satisfy the aforementioned operating parameters, Determining candidate edge layer and edge platform placements for the aforementioned workload involves: A step of determining the operational parameter ratio for each of the candidate edge layers and edge platform configurations, wherein the operational parameter ratio corresponds to the ratio of resource availability of a given edge layer and edge platform configuration that satisfies the resource requirements of the requesting edge platform; This includes the step of selecting an edge layer and edge platform arrangement having the optimal ratio of operating parameters as the candidate edge layer and edge platform arrangement. method.

20. The method according to claim 19, wherein determining the operating parameters includes deriving a subset of factors required for the operating parameters.

21. The method according to claim 20, wherein a subset of the factors includes the service level agreement (SLA) attribute of the edge platform.

22. The method according to claim 20, wherein the subset of the factors includes at least one of the security level and the accuracy level.

23. The method according to any one of claims 20 to 22, wherein determining the operating parameters includes estimating resource requirements for the requesting edge platform, the resource requirements including power measurement, thermal measurement, latency measurement, and bandwidth measurement for the workload based on a subset of the factors.

24. The method according to claim 23, further comprising the step of updating the resource requirements based on updated operating parameters.

25. Analyzing the candidate edge layer and edge platform arrangement is: Telemetry data is received from each of the candidate edge layers and edge platform deployments. This includes determining the resource availability of each of the candidate edge layer and edge platform configurations, wherein the resource availability corresponds to power availability, thermal availability, latency availability, and bandwidth availability for each of the candidate edge layer and edge platform configurations, based on the telemetry data. Determining the candidate edge layer and edge platform placement for the aforementioned edge platform is: A step of determining the operational parameter ratio for each of the candidate edge layers and edge platform configurations, wherein the operational parameter ratio corresponds to the ratio of resource availability for a given edge layer and edge platform configuration that satisfies the resource requirements of the edge platform; This includes the step of selecting an edge layer and edge platform arrangement having the optimal ratio of operating parameters as the candidate edge layer and edge platform arrangement. The method according to any one of claims 19 to 24.

26. The method of claim 25, further comprising adjusting resource requirements and cost function weights in response to the operating parameter ratios for each of the candidate edge layers and edge platform placements not meeting a threshold.

27. A non-temporary computer-readable medium containing instructions for performing the method according to any one of claims 19 to 24.

Images