DYNAMIC PARALLEL PROCESSING IN AN EDGE COMPUTING SYSTEM

Abstract

Data to be processed by a particular service executed by a first edge computing device in an application is analyzed to determine characteristics of the data. An opportunity to replicate the particular service on multiple edge computing devices is determined based on characteristics of the data. It is determined that a second edge computing device is available to execute a replicated instance of the particular service. Replication of the particular service is initiated on multiple edge computing devices that include the second edge computing device. An output of an instance of the particular service executed on the first edge computing device and an output of the replicated instance of the particular service executed on the second edge computing device are combined to form a single output for the particular service.

Description

TECHNICAL FIELD

[0001] This disclosure relates generally to the field of computer networking and, more particularly, though not exclusively, to establishing parallel processing pipelines in edge computing networks. BACKGROUND

[0002] Edge computing, including mobile edge computing, can provide application developers and content providers with cloud computing capabilities and an information technology service environment at the edge of a network. Edge computing can offer several advantages compared to traditional centralized cloud computing environments. For example, edge computing can provide a user equipment (UE) with a service with lower latency, lower cost, higher bandwidth, closer proximity, or exposure to a real-time wireless network and contextual information. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The present disclosure is best understood from the following detailed description when read with the accompanying figures. It should be emphasized that, in accordance with standard industry practice, various features are not necessarily drawn to scale and are used for illustration purposes only. Where a scale is shown, it is explicitly or implicitly intended to provide an illustrative example only. In other embodiments, the dimensions of the various features may be arbitrarily exaggerated or reduced for clarity of discussion. Fig. Figure 1 illustrates an overview of an edge cloud configuration for edge computing. Fig. 2 illustrates operational layers among endpoints, an edge cloud, and cloud computing environments. Fig. Figure 3 illustrates an example approach for networking and services in an edge computing system. Fig. Figure 4 illustrates a block diagram for an example edge computing device. Fig. Figure 5 illustrates an overview of distributed computing layers deployed in an edge computing system. Fig. Figure 6 illustrates a block diagram of an example end-to-end application using an edge computing system. Fig. Figure 7 illustrates a block diagram of an example end-to-end application using an edge computing system including dynamic replication of services. Fig. Figure 8 illustrates a block diagram of an example edge computing system that includes data scaling operations. Fig. Figure 9 illustrates a block diagram of an example edge computing system that includes multiple data preprocessing stages. Fig. 10 illustrates a block diagram of an example computing system that includes dynamic replication of services. Fig. Figure 11 is a simplified flowchart illustrating an example technique for starting parallel data processing pipelines. Fig. Figure 12 is a simplified block diagram showing an example implementation of an application that uses dynamic replication of services. Fig. Figure 13 is a flowchart showing an example implementation of an application that uses dynamic replication of services. Fig. Figure 14 is a flowchart showing a specific example of an application that uses dynamic replication of services. EMBODIMENTS OF THE REVELATION

[0004] The following disclosure provides numerous different embodiments or examples for implementing various features of the present disclosure. Specific examples of components and arrangements are described below to simplify the present disclosure. These are, of course, merely examples and are not to be considered limiting. Furthermore, reference numerals and / or letters may be repeated throughout the various examples of the present disclosure. This repetition is for simplicity and clarity and is not intended, in itself, to prescribe any relationship between the various embodiments and / or configurations discussed. Different embodiments may have different advantages, and no particular advantage is necessarily required of every embodiment.

[0005] Fig. 1 is a block diagram 100 showing an overview of an edge computing configuration that includes a processing layer, referred to as an "edge cloud" or "edge system" in many of the following examples. As shown, the edge cloud 110 is co-located at an edge location, such as an access point or base station 140, a local processing hub 150, or a central office 120, and thus may include multiple entities, devices, and equipment instances. The edge cloud 110 is located much closer to the endpoint (consumer and producer) data sources 160 (e.g., autonomous vehicles 161, user devices 162, business and industrial devices 163, video capture devices 164, drones 165, smart city and building devices 166, sensors and IoT devices 167, etc.) than the cloud data center 130.Compute, memory, and storage resources offered at the edges in the edge cloud 110 can be used to provide ultra-low latency response times for services and functions used by the endpoint data sources 160, as well as to reduce network backhaul traffic from the edge cloud 110 to the cloud data center 130, thereby improving energy consumption and overall network utilization, among other benefits.

[0006] Computing, memory, and storage are scarce resources and generally decrease with edge location (e.g., fewer processing resources are available at consumer endpoint devices than at a base station than at a central office). However, the closer the edge location is to the endpoint (e.g., end user equipment (UE)), the more space and power are often constrained. Thus, edge computing attempts to reduce the resources required for network services by distributing more resources that are closer both geographically and in network access time. In this way, edge computing attempts to bring compute resources to the working data where appropriate, or bring the working data to the compute resources.

[0007] For example, edge gateway servers can be equipped with memory pools and storage resources to perform real-time computations for low-latency use cases (e.g., autonomous driving or video surveillance) for connected client devices. Or, as one example, base stations can be enhanced with compute and acceleration resources to directly process service workloads for connected end devices without further communicating data across feeder networks. As another example, the network management hardware at the headquarters can be replaced by standardized compute hardware that executes virtualized network functions and provides compute resources for running services and consumer functions for connected devices. Within edge computing networks, there may be scenarios in services where the computation resource is "offloaded" to the data (e.g.,migrated), as well as scenarios where the data is "moved" to the compute resource. Or, as one example, the base station's compute, acceleration, and network resources can provide services to scale to workload needs on an as-needed basis, by activating dormant capacity (contribution, on-demand capacity) to manage exceptions or emergencies, or to provide longevity for deployed resources over a significantly longer deployed lifecycle.

[0008] Fig. Figure 2 illustrates operational layers among endpoints, an edge cloud, and cloud computing environments. In particular, Fig. 2 illustrates examples of computation use cases 205 that utilize the edge cloud 110 under several illustrative layers of network computation. The layers begin at an endpoint layer (devices and things layer) 200, which accesses the edge cloud 110 to perform data ingestion, analytics, and data consumption activities. The edge cloud 110 may span multiple network layers, such as an edge device layer 210 with gateways, on-premises servers, or network devices (nodes 215) located in physically proximate edge systems; a network access layer 220 comprising base stations, radio processing units, network hubs, regional data centers (DCs), or local network devices (devices 225); and any devices, devices, or nodes located in between (in layer 212, not illustrated in detail).The network communications within the edge cloud 110 and among the various layers may occur over any number of wired or wireless media, including connectivity architectures and technologies not shown.

[0009] Examples of latency resulting from network communication distance and processing time constraints can range from less than one millisecond (ms) when within the endpoint layer 200, to less than 5 ms at the edge device layer 210, to even between 10 and 40 ms when communicating with nodes at the network access layer 220. Beyond the edge cloud 110 are core network 230 and cloud data center 240 layers, each with increasing latency (e.g., between 50 to 60 ms at the core network layer 230 to 100 or more ms at the cloud data center layer). As a result, operations in a core network data center 235 or a cloud data center 245 with latencies of at least 50 to 100 ms or more will not be able to implement many time-critical functions of use cases 205.Each of these latency values is provided for illustrative and contrast purposes; it is understood that the use of other access network media and technologies can further reduce latencies. In some examples, respective portions of the network may be categorized as "close-edge," "local-edge," "near-edge," "middle-edge," or "far-edge" layers with respect to a network source and destination.For example, from the perspective of the core network data center 235 or a cloud data center 245, a central office or content data network may be considered to be located within a "near-edge" layer (close to the cloud, with high latency when communicating with the devices and endpoints of the use cases 205), whereas an access point, base station, on-premises server, or network gateway may be considered to be located within a "far-edge" layer (away from the cloud, with low latency when communicating with the devices and endpoints of the use cases 205).It is understood that other categorizations of a particular network layer as forming a "close," "local," "near," "middle," or "far" edge may be based on latency, distance, number of network hops, or other measurable characteristics as measured by a source in any of the network layers 200-240.

[0010] The various use cases 205 may access resources under pressure from incoming flows due to multiple services utilizing the edge cloud. To achieve low-latency results, the services executing within the edge cloud 110 balance varying requirements in terms of: (a) priority (throughput or latency) and quality of service (QoS) (e.g., traffic for an autonomous car may have a higher priority than a temperature sensor in terms of response time requirement; or a performance sensitivity / bottleneck may exist at a compute / accelerator, memory, storage, or network resource depending on the application); (b) reliability and resilience (e.g.,some input flows must be processed and traffic routed with mission-critical reliability, whereas some other input flows can tolerate occasional failure depending on the application); and (c) physical constraints (e.g., power, cooling, and form factor, etc.); and (d) contractual agreements (e.g., service level agreements (SLAs) or service level objectives (SLOs)).

[0011] The end-to-end service view for these use cases incorporates the concept of a service flow and is associated with a transaction. The transaction displays the details of the entire service request for the entity consuming the service, as well as the associated services for the resources, workloads, workflows, and requirements for the trading and trading layer functions. The services, which execute with the described "conditions," can be managed at each layer in a way that ensures real-time and runtime contract compliance for the transaction throughout the service lifecycle.If a component in the transaction fails to meet its agreed-upon service level agreement (SLA), the system as a whole (components in the transaction) can provide the capability to (1) understand the impact of the SLA violation, (2) augment other components in the system to reinstate the overall transaction SLA, and (3) implement steps to mitigate the situation.

[0012] Accordingly, taking these variations and service characteristics into account, edge computing within edge cloud 110 can provide the ability to serve and respond to multiple applications of use cases 205 (e.g., object tracking, video surveillance, connected cars, etc.) in real-time or near-real-time, and meet ultra-low latency requirements for these multiple applications. These benefits enable a whole new class of applications (e.g., virtual network functions (VNFs), Function-as-a-Service (FaaS), Edge-as-a-Service (EaaS), standard processes, etc.) that cannot utilize traditional cloud computing due to latency or other limitations.

[0013] However, the benefits of edge computing also come with the following caveats. Devices at the edge are often resource-constrained, putting pressure on edge resource utilization. This is typically addressed by pooling memory and storage resources for use by multiple users (tenants) and devices. The edge may be power- and cooling-constrained, so the power consumption by the most power-consuming applications must be considered. There may be inherent power delivery trade-offs in these pooled storage resources, as many are likely to use emerging storage technologies where more performance requires greater memory bandwidth.Likewise, enhanced hardware security and trusted functions of a chain of trust are also required, as edge locations may be unmanned and may even require appropriate permissions for access (e.g., if located at a third-party site). Such issues may be magnified in the edge cloud 110 in a multi-tenant, multi-owner, or multi-access situation where services and applications are requested by many users, especially as network usage fluctuates dynamically and the composition of the multiple stakeholders, use cases, and services changes.

[0014] At a more generic level, an edge computing system can be described as comprising any number of deployments at the previously discussed layers operating in the edge cloud 110 (network layers 200-240) that provide coordination of client and distributed computing devices. One or more edge gateway nodes, one or more edge aggregation nodes, and one or more core data centers may be distributed across layers of the network to provide an implementation of the edge computing system by or on behalf of a telecommunications service provider ("telco," "TSP," or communications service provider (CmSP)), Internet of Things service provider, cloud service provider (CSP), an enterprise entity, or any number of other entities.Different implementations and configurations of the edge computing system can be dynamically deployed, such as when they are orchestrated to meet service objectives.

[0015] Consistent with the examples provided herein, a client compute node may be any type of endpoint component, device, appliance, or other capable of communicating as a producer or consumer of data. Further, the term "node" or "device" as used in the edge computing system does not necessarily imply that such node or device operates in a client or agent / minion / follower role; rather, any of the nodes or appliances in the edge computing system refer to individual entities, nodes, or subsystems that include discrete or interconnected hardware or software configurations to enable or utilize the edge cloud 110.

[0016] As such, the edge cloud 110 is comprised of network components and functional features operated by and within edge gateway nodes, edge aggregation nodes, or other edge compute nodes under network layers 210-230. The edge cloud 110 may thus be implemented as any type of network that provides edge computing and / or storage resources located near radio access network (RAN)-enabled endpoint devices (e.g., mobile computing devices, IoT devices, smart devices, etc.) discussed herein. In other words, the edge cloud 110 can be thought of as an "edge" connecting the endpoint devices and traditional network access points that serve as an entry point into service provider core networks, including mobile carrier networks (e.g., Global System for Mobile Communications (GSM) networks, Long-Term Evolution (LTE) networks, 5G / 6G networks, etc.).), while also providing storage and / or computation capabilities. Other types and forms of network access (e.g., Wi-Fi, long-range wireless networks, wired networks, including optical networks) may also be used instead of or in combination with such 3GPP bearer networks.

[0017] In Fig. 3, various client endpoints 310 (in the form of mobile devices, computers, autonomous vehicles, enterprise computing equipment, industrial processing equipment) exchange requests and responses specific to the type of endpoint network aggregation. For example, client endpoints 310 may obtain network access over a wired broadband network by exchanging requests and responses 322 through an on-premises network system 332. Some client endpoints 310, such as mobile computing devices, may obtain network access over a wireless broadband network by exchanging requests and responses 324 through an access point (e.g., cellular tower) 334. Some client endpoints 310, such as autonomous vehicles, may obtain network access for requests and responses 326 over a wireless vehicular network through a roadside network system 336.However, regardless of the type of network access, the TSP may deploy aggregation points 342, 344 within the edge cloud 110 to aggregate traffic and requests. Thus, the TSP may deploy various computing and storage resources within the edge cloud 110, such as at edge aggregation nodes 340, to deliver requested content. The edge aggregation nodes 340 and other systems of the edge cloud 110 are connected to a cloud or data center 360, which uses a backhaul network 350 to fulfill higher latency requests from a cloud / data center for websites, applications, database servers, etc. Additional or consolidated instances of the edge aggregation nodes 340 and the aggregation points 342, 344, including those deployed on a single server framework, may also exist within the edge cloud 110 or other areas of the TSP infrastructure.

[0018] Fig. 4 is a block diagram of an example of components that may be present in an example edge computing device 450 for implementing the techniques described herein. The edge device 450 may include any combination of the components shown in the example or mentioned in the above disclosure. The components may be implemented as integrated circuits, blocks of intellectual property, portions thereof, discrete electronic devices or other modules, logic, hardware, software, firmware, or a combination thereof in the edge device 450, or as components otherwise integrated into a housing of a larger system. Furthermore, the block diagram is intended to Fig. 4 shows an overview of the components of the edge device 450. However, some of the components shown may be omitted, additional components may be present, and the arrangement of the components shown may be different in other embodiments.

[0019] The edge device 450 may include processor circuitry, for example, in the form of a processor 452, which may be a microprocessor, a multi-core processor, a multi-threaded processor, an ultra-low-voltage processor, an embedded processor, or other known processing elements. The processor 452 may be part of a system-on-chip (SoC), in which the processor 452 and other components are housed in a single integrated circuit or package. The processor 452 may communicate with a system memory 454 via an interconnect 456 (e.g., a bus). Any number of memory devices may be used to provide a given amount of system memory. To provide persistent storage of information, such as data, applications, operating systems, and so on, a storage 458 may also be coupled to the processor 452 via circuitry 456.In one example, storage 458 may be implemented using a solid-state disk drive (SSDD). Other devices that may be used for storage 458 include flash memory cards, such as SD cards, microSD cards, xD-Picture cards, and the like, as well as USB flash drives. In low-power implementations, mass storage 458 may be on-die memory or a register connected to processor 452. However, in some examples, storage 458 may be implemented using a micro hard disk drive (HDD). Further, any number of new technologies may be used for storage 458 in addition to or in place of the described technologies, such as, but not limited to, resistive change memory, phase change memory, holographic memory, or chemical memory.

[0020] The components can communicate via interconnect 456. Interconnect 456 can incorporate any number of technologies, including PCI Express (PCIe), Compute Express Link (CXL), NVLink, HyperTransport, or any number of other technologies. Interconnect 456 can be a proprietary bus used, for example, in an SoC-based system. Other bus systems can be included, such as an I2C interface, an SPI interface, point-to-point interfaces, and a power bus, to name a few.

[0021] Given the variety of types of applicable communications from the device to another component or network, applicable communication circuitry employed by the device may include or be implemented by any one or more of components 462, 466, 468, or 470. Accordingly, in various examples, applicable means for communicating (e.g., receiving, transmitting, etc.) may be implemented by such communication circuitry. For example, interconnect 456 may couple processor 452 to a mesh transceiver 462 for communication with other mesh devices 464. Mesh transceiver 462 may use any number of frequencies and protocols, e.g., 2.4 gigahertz (GHz) transmissions according to the IEEE 802.15.4 standard, Bluetooth ® Low Energy (BLE) standard, as developed by Bluetooth ®Special Interest Group, or the ZigBee ® standard, to name a few. The Mesh Transceiver 462 can communicate using multiple standards or radios for communications at different ranges.

[0022] A wireless network transceiver 466 may be included to communicate with devices or services in the cloud 400 via local or wide area network protocols. For example, the edge device 450 may communicate over a wide area using LoRaWAN™ (Long Range Wide Area Network), among other example technologies. Indeed, any number of other radio communications and protocols may be used in addition to the systems mentioned for the mesh transceiver 462 and wireless network transceiver 466 as described herein. For example, the radio transceivers 462 and 466 may include an LTE or other cellular transceiver that uses spread spectrum communication (SPA / SAS) to implement high-speed communication. Furthermore, any number of other protocols may be used, such as Wi-Fi ®Networks for medium-speed communications and providing mesh communications. A network interface controller (NIC) 468 may be included to provide wired communication with the cloud 400 or with other devices, such as the mesh devices 464. The wired communication may provide an Ethernet connection or may be based on other types of networks, protocols, and technologies.

[0023] The interconnect 456 may couple the processor 452 to an external interface 470 used for connecting external devices or subsystems. The external devices may include sensors 472, such as accelerometers, level sensors, flow sensors, optical light sensors, camera sensors, temperature sensors, global positioning system (GPS) sensors, pressure sensors, barometric pressure sensors, and the like. The external interface 470 may also be used to connect the edge device 450 to actuators 474, such as circuit breakers, valve actuators, an audible tone generator, a visual warning device, and the like.

[0024] In some optional examples, various input / output (I / O) devices may be present within or connected to the edge device 450. Further, some edge computing devices may be battery-operated and include one or more batteries (e.g., 476) to power the device. In such cases, the edge device 450 may include a battery monitor / charger 478 to monitor the state of charge (SoCh) of the battery 476. The battery monitor / charger 478 may be used to monitor other parameters of the battery 476 to provide failure predictions, such as the state of health (SoH) and state of function (SoF) of the battery 476, which, among other example uses, may trigger an edge system to deploy other hardware (e.g., in the edge cloud or a nearby cloud system) to supplement or replace a device that is failing power.In some cases, the device 450 may also or instead include a power block 480, or another power supply coupled to a grid may be coupled to the battery monitor / charger 478 to charge the battery 476. In some examples, the power block 480 may be replaced with a wireless power receiver to receive power wirelessly, e.g., via a loop antenna in the edge device 450, among other examples.

[0025] Storage 458 may include instructions 482 in the form of software, firmware, or hardware commands to implement the workflows, services, microservices, or applications to be executed in transactions of an edge system, including techniques described herein. Although such instructions 482 are shown as blocks of code contained in memory 454 and storage 458, it should be understood that any of the blocks of code may be replaced by hard-wired circuitry, for example, incorporated in an application-specific integrated circuit (ASIC). In some implementations, hardware of edge computing device 450 (separately or in combination with instructions 488) may configure the execution or operation of a trusted execution environment (TEE) 490.In one example, the TEE 490 operates as a protected area accessible by the processor 452 to enable secure access to data and secure execution of instructions, among other example features.

[0026] At a more general level, an edge computing system can be described as comprising any number of deployments operating in an edge cloud 110 that provide coordination of client and distributed computing devices. Fig. 5 provides a more abstracted overview of distributed computing layers used for illustrative purposes in an edge computing environment. For example, Fig. 5 generically illustrates an edge computing system for providing edge services and applications to multi-stakeholder entities, as distributed among one or more client compute nodes 502, one or more edge gateway nodes 512, one or more edge aggregation nodes 522, one or more core data centers 532, and a global network cloud 542, as distributed across layers of the network. The implementation of the edge computing system may be deployed in or on behalf of a telecommunications service provider ("Telco" or "TSP"), Internet of Things service provider, cloud service provider (CSP), an enterprise entity, or any number of other entities.

[0027] Each node or device of the edge computing system is located at a specific layer corresponding to layers 510, 520, 530, 540, 550. For example, the client computing nodes 502 are each located at an endpoint layer 510, while each of the edge gateway nodes 512 is located at an edge device layer 520 (local level) of the edge computing system. Additionally, each of the edge aggregation nodes 522 (and / or fog devices 524, if arranged or operating with or under a fog networking configuration 526) is located at a network access layer 530 (an intermediate level). Fog computing (or "fogging") generally refers to extensions of cloud computing to the edge of an enterprise network, typically in a coordinated distributed or multi-node network.Some forms of fog computing involve the provision of compute, storage, and networking services between end devices and cloud computing data centers on behalf of the cloud computing sites. Such forms of fog computing provide operations consistent with edge computing as discussed herein; many of the edge computing aspects discussed herein are applicable to fog networks, fogging, and fog configurations. Furthermore, aspects of the edge computing systems discussed herein may be configured as a fog, or aspects of a fog may be integrated into an edge computing architecture.

[0028] The core data center 532 is located at a core network layer 540 (e.g., a regional or geographically central level), while the global network cloud 542 is located at a cloud data center layer 550 (e.g., a national or global level). The use of "core" is intended to refer to a centralized network location deeper within the network that can be accessed by multiple edge nodes or components; however, a "core" does not necessarily denote the "midpoint" or deepest location of the network. Accordingly, the core data center 532 may be located within, at, or near the edge cloud 110.

[0029] Although an illustrative number of client compute nodes 502, edge gateway nodes 512, edge aggregation nodes 522, core data centers 532, global network clouds 542 in Fig. 5, it is understood that the edge computing system may include more or fewer devices or systems at each layer. In addition, as shown in Fig. 5, the number of components of each layer 510, 520, 530, 540, 550 generally increases at each lower level (i.e., as they are moved closer to the endpoints). Therefore, one edge gateway node 512 may serve multiple client compute nodes 502, and one edge aggregation node 522 may serve multiple edge gateway nodes 512.

[0030] Consistent with the examples provided herein, each client compute node 502 may be implemented as any type of endpoint component, device, appliance, or other "thing" capable of communicating as a producer or consumer of data. Further, the term "node" or "device" as used in the edge computing system 500. Thus, the edge cloud 110 is formed from network components and functional features operated by and within the edge gateway nodes 512 and the edge aggregation nodes 522 of layers 520, 530, respectively. The edge cloud 110 may be implemented as any type of network providing edge computing and / or storage resources located near radio access network (RAN)-enabled endpoint devices (e.g., mobile computing devices, IoT devices, smart devices, etc.) deployed in Fig. 5 as the client compute nodes 502. In other words, the edge cloud 110 can be thought of as an "edge" that connects the endpoint devices and traditional mobile network access points that serve as an entry point into service provider core networks, including bearer networks (e.g., GSM (Global System for Mobile Communications) networks, LTE (Long-Term Evolution) networks, 5G networks, etc.), while also providing storage and / or compute capabilities. Other types and forms of network access (e.g., Wi-Fi, long-range wireless networks) may also be used instead of or in combination with such 3GPP bearer networks.

[0031] In some examples, the edge cloud 110 may form a portion of, or otherwise provide an entry point into or across, a fog network configuration 526 (e.g., a network of fog devices 524, not shown in detail), which may be implemented as a horizontal and distributed system-level architecture that distributes resources and services to perform a specific function. For example, a coordinated and distributed network of fog devices 524 may perform computing, storage, control, or networking aspects in the context of an IoT system arrangement. Other networked, aggregated, and distributed functions may exist in the edge cloud 110 between the cloud data center layer 550 and the client endpoints (e.g., client compute nodes 502).

[0032] The edge gateway nodes 512 and the edge aggregation nodes 522 cooperate to deliver various edge services and security to the client compute nodes 502. Because each client compute node 502 may be stationary or mobile, each edge gateway node 512 may cooperate with other edge gateway devices to propagate currently deployed edge services and security as the corresponding client compute node 502 moves within a region. To this end, each of the edge gateway nodes 512 and / or edge aggregation nodes 522 may support multi-tenant, multi-stakeholder configurations in which services from (or hosted for) multiple service providers and multiple consumers may be supported and coordinated across a single or multiple computing devices.

[0033] Edge computing systems, such as those introduced above, can be used to provide hardware and logic to implement various applications and services. As shown, for example, in the simplified block diagram 600 of Fig. 6, an application 610 may be defined to perform various operations on various data (e.g., 605) provided as an input to the application. In conventional edge applications, a chain of one or more services (e.g., 620a-620n) may operate on the data to implement the end-to-end application. For example, a first service (e.g., 620a) may be programmatically configured to perform a first set of operations on the data 605 to transform the data and / or derive additional data (e.g., metadata, inference results, identifiers or labels, etc.). The output of the first service 620a may be provided as an input to another service in the chain, which may operate on that output to produce another output, and so on, until a final output 630 of the final service 620n in the chain is produced for the application.This output may be provided to another application, data store, or system (e.g., 635). The various services 620a-n in the application may execute on one or more edge computing devices in a distributed edge network 625. In some cases, a single edge computing device may execute each of the services in the service chain, although advantageously, multiple edge computing devices in other instances may be used to execute the multiple services. In conventional edge computing systems, a single processing pipeline 615 is implemented, such as in the example of FIG. Fig. 6, where the application is predefined to operate on specific expected data (e.g., 605) according to a customized designed pipeline with scheduling policies and compute resource allocation optimized at the local edge device level.

[0034] Although edge-based application implementations can effectively manage edge resources and meet the defined service level requirements or agreements (e.g., SLAs) for the application, such applications may not be able to dynamically maintain such performance when variances and complexities arise in the data to be processed. For example, in the example of Fig. 6, the application 610 may be configured to identify a person in image data as well as certain characteristics of the person (e.g., predicted age, gender, identification, gestures, location within a scene, etc.). A chain of services 620a-n may, in one example, be configured to determine such characteristics when a single person is presented within the image, but may, for example, be ill-equipped when multiple (and varying numbers of) people are included in the image, and may, in some cases, among other examples, be forced to repeat the processing pipeline in a serial manner to complete the processing pipeline for each person appearing in the image. However, such an approach may violate SLA conditions (e.g., end-to-end latency), among other example issues.

[0035] Modern edge processing software architectures can follow the microservices framework, where services or microservices (e.g., each programmed to perform a specific task or subfunction) are chained together to achieve complete functionality for an end-to-end service. In an end-to-end application processing context, the higher the service requirement, the greater the resource allocation required to effectively implement the application. Therefore, the resource requirements for a larger application may overwhelm the resources of an individual edge computing system. Indeed, resources in edge systems are typically limited, although distributed edge infrastructure can be used to combine resources to implement an application or service.A distributed edge infrastructure may include a set of decentralized nodes, such as geographically distributed edge devices such as retail edge servers, cloud service provider edge instances, and traditional network provider edge systems (e.g., 5G base stations and access points, etc.), among other examples. The edge infrastructure can be made available on demand in some implementations and can be used to dynamically instantiate resources in response to the detected real-time requirements of various applications.

[0036] In an enhanced edge computing infrastructure, the efficiency of end-to-end processing can be improved in an application-specific manner and tailored to specific time and operating conditions, for example, by reducing latency, improving reliability, and optimizing resource utilization. For example, an enhanced edge computing system can implement autonomous and dynamic service replication based on the application's capabilities and SLA and the characteristics of the specific input data to be operated on by the application's service chain. Service replication causes multiple instances of a given service or microservice to be instantiated on multiple edge systems. Service replication can provide higher processing resources across different nodes or locations within edge networks, leveraging characteristics of a distributed edge infrastructure.

[0037] As an example, an application may be developed to perform video processing for intrusion detection, where processing speed is important (e.g., to detect intrusions in a timely manner) and where resources may be challenged during peak demand periods (e.g., during rush hour, periods or days of high pedestrian traffic, etc.). The enhanced edge computing infrastructure may dynamically identify opportunities to instantiate concurrent procedures in a processing pipeline for an application. For example, an enhanced edge computing infrastructure may utilize dynamic service replication and segmentation capabilities based on characteristics of data to be processed in the pipeline, among other exemplary features.

[0038] An improved edge computing infrastructure, as discussed here, can enable the adoption of real-time processing for critical applications such as telemedicine, industrial robotics, and fraud detection, where real-time analytics are essential. Existing solutions typically rely on load balancing a service by horizontally and vertically scaling microservice instances based on incoming requests and local resource utilization. For example, existing technologies may rely on developer tools such as GStreamer and DLStreamer, which require a developer to predict and pre-build any parallelization during development time and implement appropriate code, command lines, or scripts to create a dedicated single processing pipeline used throughout the application's lifetime.This approach places a burden on developers and IT management regarding the deployment of edge services on edge infrastructures. In an improved edge computing infrastructure, the end-to-end application context of an application can be considered, preprocessing data to be consumed by the application based on the data type and the type of analytics and operations to be performed, as the basis for dynamically scaling microservices relative to the data context through adaptive service replication. Through such replication and / or segmentation, multiple parallelized processing pipelines can be instantiated to exploit edge resource availability in distributed networks, with the outputs of these multiple service instances postprocessed to combine the results to meet end-to-end service requirements.

[0039] Referring to Fig. 7 shows a simplified block diagram 700 of an example edge computing system for implementing an end-to-end application leveraging a distributed edge network, where the edge computing devices include logic (e.g., implemented in hardware and / or software) to implement adaptive multi-pipeline processing (or replicated service chaining) to more effectively utilize the resources of the distributed edge network and increase the efficiency of the application's performance, including instances of complex input data (e.g., 605). In this example, the entire end-to-end application pipeline may be modified to include data scaling preprocessing 710 to dynamically parallelize the service chain 715 through adaptive mule pipelining 720.A data aggregation stage 730 may also be included to aggregate and combine the component results of parallelized services to produce an expected output (e.g., for further processing by a next service in the chain or by another application (e.g., 635) that is to consume the output of application 610). For example, in one example, an application may be configured to identify specific individuals in image or video data fed as an input to the application (e.g., for use in a security, retail analytics, or other application).

[0040] Input processing 705 of input data 605 may be performed (e.g., by one of the edge computing devices in the distributed edge network) in conjunction with the application to identify whether there are opportunities to replicate and parallelize instances of a next service in the application's service chain. It may be expected that in some cases, the service or workload can be conveniently and effectively executed using a single edge computing device for certain types of data inputs. However, other data inputs to the service may reveal, through preprocessing of the data input, that an opportunity exists to replicate the service on one or more additional edge computing devices. For example, in the case of image data and a service for performing image processing on the image data, the input processing may determine whether one or more people appear in the data.In this example, multiple people are identified as appearing in the image, and the edge computing device responds by initiating replication of the service intended to operate on the data (e.g., "Service A") to create a number of parallel instances of the service equal to the number of people's faces appearing in the data. Replicating the service may involve identifying available candidate edge computing devices (e.g., within a similar geography within the distributed edge network) and instantiating these additional edge computing devices with instances of the service as well as their respective portion of the input data. Each instance of the service may then operate on its respective portion of the data (e.g., corresponding to a respective one of the people) to produce a respective output.Since a next service in the application's service chain may expect to operate on input data of a particular format, the edge computing system may first aggregate the respective outputs of the multiple replicated instances of Service A to produce (at 725) a single combined output that incorporates the information in the outputs of the replicated instances into the format expected by the next service in the chain.

[0041] Edge computing devices are typically resource-constrained and distributed. Therefore, highly demanding applications to be implemented on an edge network typically involve dedicated scheduling, allocation, and a single heavyweight execution pipeline to meet the requirements of a given application. Typically, this heavyweight solution is developed and maintained for the lifetime of the application. An enhanced edge computing infrastructure, as described here, can reduce the need for purpose-built heavyweight solutions by using highly dynamic multiple processing pipelines that leverage the distributed nature of the edge infrastructure.Accordingly, a preprocessing step may evaluate data to determine opportunities to create multiple pipeline processing stages (corresponding to a single or multiple services or functions within the application), with the outputs of these parallelized pipelines being post-processed, among other exemplary features, to combine the results and preserve end-to-end context.

[0042] In one implementation, application and service management may be edge-native and configured to support critical application requirements, such as large-scale computing with dedicated end-to-end service level agreements (SLAs). Service configuration may be exposed to end users through dedicated interfaces (e.g., to define end-to-end SLAs, security requirements, etc. for an application). In some implementations, a developer framework or kit may be defined and exposed to assist developers in developing applications that could leverage the creation of dynamic processing pipelines, among other example tools. Consumable interfaces for middleware integration (e.g., intermediate services) may be independently replicated in the intermediate stages of execution (e.g., where the SLA can be achieved between Stage A to Stage N or Service A to Service N).

[0043] An enhanced edge computing system may enable flexible and opportunistic scalability with the ability to dynamically increase or decrease processing capacity by decomposing services that can be placed across a network of distributed edge nodes (e.g., within a specific geographic location or boundary) to implement an independent processing pipeline. In such a system, a pool of compute resources may not need to be predefined or pre-orchestrated for an end-to-end application, with the enhanced system enabling a service function chain to create an independent processing pipeline across an independent path of distributed edge computing nodes. To achieve such flexibility, in some implementations, the system may incorporate data tagging (e.g., timestamping, sequencing, service instance stamping, etc.).) or other metadata to synchronize the output and inputs of multiple processing pipelines generated by opportunistic service replication provided by the enhanced edge computing system. Such metadata can be used, among other examples, to post-process the data in a consistent and organized manner to ensure the successful regeneration of the output in the expected format for the next service in the service chain or as the final output of the application itself.

[0044] An edge computing system may include logic (e.g., implemented in hardware and / or software) to implement dynamic replication of services within an end-to-end application. To determine an opportunity to replicate services, an edge computing system may identify the type of service (e.g., the operations or functions to be implemented in the service) as well as the input data for the service. The edge computing system may support a variety of replication strategies, as such replication strategies and techniques may vary based on the changing environment, data type, or changing end-to-end characteristics (e.g., relative demand (e.g., rush hour versus empty captures), changing network conditions, analytics load levels, and user-defined conditions to ensure end-to-end service quality, among other examples.To determine whether a replication opportunity exists and which replication strategies can be applied, the data can be preprocessed to detect attributes of the data and identify opportunities relative to the service type, replicate the service, and parallelize tasks to be performed on the data by the service. To determine the replication strategy to be implemented, data preprocessing can be performed using a centralized or localized approach and / or a distributed approach. In some cases, machine learning can be used to perform the preprocessing (e.g., using reinforcement learning, deep learning, or other techniques). In some implementations, a service replication strategy can include a data segmentation step where multiple parts of a data item can be identified.Replication may involve replicating a given service such that each replicated instance of the service operates on a respective one of the identified portions of the data element. Accordingly, in some cases, data may be segmented to create a unique processing pipeline, such as, among other examples, by considering only a group or collection of features, selecting a specific feature set, or working on a background to provide auxiliary information for the overall analysis.

[0045] An edge computing system may include logic (e.g., implemented in hardware and / or software) to implement dynamic replication of services within an end-to-end application. To determine an opportunity to replicate services, an edge computing system may identify the type of service (e.g., the operations or functions to be implemented in the service) as well as the input data for the service. The edge computing system may support a variety of replication strategies, as such replication strategies and techniques may vary based on the changing environment, data type, or changing end-to-end characteristics (e.g., relative demand (e.g., rush hour versus empty captures), changing network conditions, analytics load levels, and user-defined conditions to ensure end-to-end service quality, among other examples.To determine whether a replication opportunity exists and which replication strategies can be applied, the data can be preprocessed to detect attributes of the data and identify opportunities relative to the service type, replicate the service, and parallelize tasks to be performed on the data by the service. To determine the replication strategy to be implemented, data preprocessing can be performed using a centralized or localized approach and / or a distributed approach. In some cases, machine learning can be used to perform the preprocessing (e.g., using reinforcement learning, deep learning, or other techniques). In some implementations, a service replication strategy can include a data segmentation step where multiple parts of a data item can be identified.Replication may involve replicating a given service such that each replicated instance of the service operates on a respective one of the identified portions of the data element. Accordingly, in some cases, data may be segmented to create a unique processing pipeline, such as, among other examples, by considering only a group or collection of features, selecting a specific feature set, or working on a background to provide auxiliary information for the overall analysis.

[0046] Referring to Fig. 8, a simplified block diagram 800 is shown illustrating exemplary preprocessing and data scaling to be performed in an exemplary edge deployment of an application. Data scaling and preprocessing may include a variety of operations based on the data type and the services in the service chain (e.g., to align data scaling with opportunities to parallelize the service chain). For example, preprocessing 810 of input data (e.g., 815) may be based on a configuration 805 defined for the application based on the operations to be performed on the data by the services in the application and the expected or defined output format for results generated by the application. Configuration 805 may also define how the preprocessing and analysis of the input data is to be performed (e.g.,which algorithms or tools should be used, whether locally or with the help of a remote (e.g., cloud-based) resource, etc.), and under what conditions service replication should take place (and for which services in the service chain). In one example, data scaling may include segmentation, replication, scaling, and elimination. In the case of segmentation (e.g., 825), preprocessing the data may include identifying portions of a single data item (e.g., image, video, audio, text, database, etc.) that represent different entities or representations of information that are to be subject to operations in a given service or collection of services in the processing pipeline. In some cases, segmentation 825 of the data may enable service replication to operate on each identified segment of the data. In the case of data replication (e.g.,820), characteristics of the input data may allow the data to be replicated and input to multiple different services in parallel, with each service performing a respective operation on its copy of the data (e.g., one service identifying the gender of a face depicted in an image, another service identifying the age range of a face depicted in an image, etc.). For elimination (e.g., 830), portions of the data that contain useless, confusing, or irrelevant information may be identified during preprocessing 810 and may allow these portions to be represented as null inputs to a respective processing pipeline (e.g., 850) in a set of parallelized pipelines (e.g., 835, 840, 845, 850) generated based on the preprocessing.Opportunities to scale, resize, and otherwise transform all or certain portions of the data 815 may be identified through preprocessing (e.g., to crop the data to focus on a portion (e.g., a face, a speech sample, etc.) to be focused on in a subsequent preprocessing step). Preprocessing 810 may, among other examples, identify opportunities to both replicate services in the service chain and make the processing of data segments by respective replicated service instances more efficient (e.g., in the case of data transformations or elimination).

[0047] When a service is replicated, each instance of the service may produce a respective output from the portion of the data manipulated by the service instance. To produce a complete output, in some cases, the respective outputs of the set of service instances may be combined to form a single, unitary output that captures the aggregated outputs of the multiple service instances. One of the edge computing systems used to execute one of the replicated service instances may further include logic (e.g., implemented in hardware and / or software) to aggregate the outputs of the replicated service instances and produce a combined or aggregated output for the service and pass this output as the input to a next service within the end-to-end application, or alternatively, as the final output of the end-to-end application (e.g.,(The replicated service is the final service within a chain of services implementing the end-to-end application.) To enable the aggregation of the outputs of a replicated service, metadata or other tools can be used to maintain consistency and synchronization among the replicated service instances. In one example, among other example implementations, data is tagged and maintained with consistency among replicated instances by implementing appropriate synchronization mechanisms to ensure that data remains coherent across replicas.

[0048] In some implementations, multiple preprocessing stages and multiple service replication stages can be used when implementing an end-to-end application using a distributed edge network. In some cases, multiple data scaling operations can be performed in conjunction with a common preprocessing step for a service in a service chain. In some cases, data scaling operations can be nested. For example, Fig. 9 is a simplified block diagram 900 illustrating the use of multiple processing pipelines by replicating one or more services in a service chain of an end-to-end application (e.g., 950). An application or other data source (e.g., 610) may provide data 705 for processing by the application 950. One of the services in the service chain may be performed by a particular one of potentially multiple edge computing devices in an edge cloud or other edge system. The particular edge computing device may determine 705 (e.g., from characteristics of the service and / or data) that opportunities exist to instantiate multiple processing pipelines by replicating the service across multiple edge computing devices in parallel.In this example, the particular edge computing device may determine that the data 605 and service would benefit from selecting a particular segment 920 of the data and replicating 905 that particular portion of the data (e.g., to perform the segment 920 through multiple tasks or algorithms to be performed on the input using multiple replicated instances of the service (where each replicated instance performs only one or a different subset of the service's set of algorithms) to meet or exceed a particular quality of service or performance target specified for the application 950). In this example, nested preprocessing may further include identifying, under each replicated segment of the data, that intermediate preprocessing 910 may be performed to further segment the data 705 and dynamically parallelize the service.For example, each of the data segments 920 may contain images of multiple persons or parts subject to further processing on a person-by-person or part-by-part basis, and each replicated segment 920 may be further segmented (at 910) to process the image of the individual part, person, or other entity for processing in a respective parallel processing pipeline (e.g., 835, 840, 845, 850, etc.).

[0049] The particular edge computing device performing these preprocessing steps may determine, based on the results of the preprocessing, that a number of parallel processing pipelines should be established based on the segmentation and / or replication performed during the preprocessing, and operate to instantiate a corresponding number of replicated instances of the service that should perform work on each respective segment and / or copy from the original data (e.g., 605) input to the service. In addition, the particular edge computing device may further manage the collection of results from the multiple replicated service instances and perform post-processing 915 to produce a combined or aggregated result 725 in the format required by the next service in the application or service chain or another consumer (e.g.,of the application 635) of the result of the service is expected.

[0050] Referring to Fig. Figure 10 shows a simplified block diagram 1000 illustrating another example of a system capable of implementing an end-to-end application and leveraging dynamic parallelization of processing pipelines during execution. As in other examples, the input data (e.g., from 1045) is analyzed for possible workload parallelization specific to the end-to-end context. User applications (e.g., 1045, 1046) that include both client and cloud applications can be orchestrated with orchestrators (e.g.,1050) in the cloud and / or the edge system to specify various end-to-end QoS service requirements, approvals, and configurations for the application, to guide decisions made during preprocessing, to identify whether and what data scaling is allowed or should be performed, and what, if any, constraints should apply, to determine how services are replicated and parallel processing pipelines are initiated during application execution. In addition, such configuration information may also be used, among other examples, to guide monitoring of the setup and performance of the application (and the edge devices used to execute portions of the application), as well as to establish telemetry services.Data preprocessing policies can be generated based on the learning mechanism to identify independent tasks that can be serialized to achieve an independent processing pipeline. The generation of a processing pipeline can be tied to data processing logic, so that there are dedicated input and output pairs for each processing pipeline. The number of processing pipelines can be created and destroyed based on closed-loop control or, in some implementations, by using reinforcement learning with feedback.

[0051] In the example of Fig. 10, the configuration parameters of the applications 1045 and 1046 and the characteristics of the data input to the application 1045 may be the basis for preprocessing the data and determining that replication of one or more services in the application should be performed (e.g., each of the replicated services is tasked with operating on a particular segment or copy of the data, as determined during preprocessing). Accordingly, multiple processing pipelines (e.g., 1005, 1010, 1015) may be established. In some cases, multiple stages of multiple processing pipelines may be established, with two or more services replicated in the application service chain (e.g.,with the same or different number of replicated instances and processing pipelines) based on the type of service and the data operated on by the service (which may be based on the output of a previous service in the service chain). When services are replicated, additional edge computing devices may be deployed in the same or different geographical locations to handle a respective replicated instance of a service. Accordingly, groups of edge computing devices (e.g., 1020, 1025, 1030, 1035, etc.) may be deployed to implement respective parallel processing pipelines (e.g., 1005, 1010, 1015). When deploying additional edge computing devices for use in handling replicated service instances, an edge control and location service system (e.g.,1054) may be used to identify 1060 available edge computing devices based on their location (e.g., within a certain geographic proximity of the edge computing device tasked with performing the service, and potentially also handling pre- and post-processing associated with dynamic parallelization of the service) and possessing the necessary hardware or functionality to run an instance of the particular service to be replicated. This information may be passed (e.g., at 1065) to placement and replication logic 1055 (e.g., a program executing on the edge computing device that performs the preprocessing, or another edge or cloud system resource) and used to allocate edge computation for replicated services for the application (e.g., 1070).

[0052] Referring to Fig. 11 shows a simplified flowchart 1100 illustrating an example technique for opportunistically and dynamically replicating services within an end-to-end cloud application. Data may be accessed (e.g., received) by an edge computing device that is to perform a service or task in an end-to-end application to be implemented in an edge cloud. The data may be preprocessed 1110 to identify opportunities to scale the data in some way based on characteristics of the data detected or inferred from the preprocessing (e.g., detecting certain entities, characteristics, or patterns within the input data). In some cases, multiple stages of preprocessing may be performed 1115 (e.g., multiple stages of segmenting the data, replicating followed by segmentation, segmenting followed by replication, etc.).Based on the preprocessing, opportunities (at 1120) may be determined to replicate the service and process the data using the edge computing device and one or more additional edge computing devices. The new processing pipelines may be assigned using one or more available and qualified (e.g., based on geography and / or functionality) edge computing resources 1125. In cases where multiple processing pipelines have been generated, postprocessing 1130 may be performed to combine the respective outputs generated by the replicated service instances executed by the multiple edge computing devices to produce a unitary or aggregated output 1135 in the expected format.The use of the multiple parallel processing pipelines may, for example, be monitored by an orchestrator or other utility of the edge deployment to evaluate 1140 characteristics of the execution of the end-to-end application. Feedback data may be generated based (1145) on whether or not requirements or SLAs of the application were met, or even exceeded, based on the use of the multiple processing pipelines dynamically adopted during the execution of the service. This information may be used, among other example uses, in connection with future preprocessing (e.g., 1110) of data in connection with future instances of the service, as well as future determinations of whether to launch multiple processing pipelines in connection with the service (e.g.,at 1150) to improve the system's use of such dynamically replicated service instances in future executions.

[0053] When instantiating multiple replicated service instances, input data can be distributed and shared with the other edge devices that have logic assigned to execute the replicated service instances. Fig. Figure 12 is a simplified block diagram illustrating an example software architecture for enabling such data distribution. In some cases, the data pipeline data structures are the data plane of the edge system discussed here. The data pipeline can enable the flow of messages from a producer (e.g., 1202a-b) or source of input data to distribute appropriate data to worker processes (e.g., 1245, 1250, 1255, 1260) responsible for processing the data in conjunction with the logic of the associated service. In some implementations, the distribution of data can occur according to a publish-subscribe model. For example, each pipeline service instance can have all the code or logic to execute all of these data structures, but the control configuration instructs each instance which processes it should utilize. For example, the input data can be sent to a receiver 1210 (e.g.,posted on a particular edge computing resource), and preprocessing of the data may result in different groupings (e.g., 1215, 1220) being identified to correspond to different segments or copies determined for the data based on the preprocessing. In conjunction with the assignment of multiple processing pipelines to handle these data groupings, respective consumer entities (e.g., 1225, 1230, 1235, 1240) may retrieve their respective data (e.g., generated during preprocessing) and pass it to associated worker elements (e.g., 1245, 1250, 1255, 1260), which associated handlers (e.g., 1265, 1270, 1275, 1280) may utilize to provision and manage the respective system resources for operating on the data. Fig. 13 illustrates an example flow and messaging in distributing data (e.g., generated by a camera data source 610) from the producer (e.g., 1202) to the worker 1245 and the handler 1265 according to an example implementation.

[0054] In some implementations, control plane data structures may be provided to enable remote procedure calls to processes through a centralized mechanism. These remote procedure calls may include commands for control, configuration, telemetry, and notifications. In one example, the interface uses HTTP web protocols and may be implemented using WebSockets and an ASP.NET SignalR core to leverage the associated rich capabilities, high performance, scalability, and community support. Such control plane data structures may include control (start, stop, restart, ping), configuration (applying settings), telemetry (logging, monitoring, observability), and notifications (process online, process offline, process error), among other examples.

[0055] Fig. Figure 14 is a simplified flowchart illustrating exemplary replication of a service. For example, in an exemplary application, a full image from a camera feed may be replicated across one or more stages in a service-chained function (e.g., toward one of the edge-based service nodes). Replica service nodes may be assigned and distributed to portions of the data for transmission to dynamically scale the service based on the requirements of end-to-end application inference, quality and accuracy of results, adaptation to network availability, or resource availability to achieve end-to-end characteristics (e.g., latency and workload requirements). In the example of Fig. 14 shows an example pipeline for face recognition processing. The pipeline may accept as an initial input image data 1405 from a camera within an environment (e.g., in close geographical proximity to an edge cloud invoked to implement the pipeline). Some of the services may include a face detection service 1410, which may take the full-size image from the camera as input and find one or more distinct faces depicted in the image. A face tag / face recognition service 1415 may also be provided, which may perform two sequential inferences on each face detected in the image (e.g., by the face detection service 1410) to identify facial landmarks and identify the face based on those landmarks.An age attribution service 1420 may also be included in the pipeline, which can take a single facial image and derive the age of the person behind the face. Additionally, an image resizing service 1425 may be provided, which takes a full-size image from the camera and scales it (e.g., to minimize network usage for a visualizer program 1430 that is intended to receive and consume the output of the pipeline).

[0056] Continuing with the example from Fig.14, in this example image processing pipeline, the minimum end-to-end processing is an image that does not include any faces. In such a case, the full image is transmitted twice: once for retrieval from the camera and once for transmission to the face detection service 1410. The output in this case would be a scaled version of the image (as generated by the image resize service 1425), which is transmitted to the visualizer application 1430. On the other hand, the maximum end-to-end processing involves an input image that includes multiple faces. The full image is transmitted twice: once for retrieval from the camera and once for transmission to the face detection service 1410.Dynamic service replication can be employed in this case, where the image is segmented to correspond to each identified face, and a replication of the face identification service and the age determination service is instantiated for each identified face segment. By transmitting each face to potentially multiple worker instances of these services (e.g., 1415, 1420), a high degree of parallelism is achieved, but at the expense of network utilization (e.g., to transmit each of the face images). These and other factors can be monitored by the system, collecting performance telemetry to optimize worker allocation based on face count versus inference compute utilization, among other exemplary features. The results of the parallel service replicas can be combined to produce a single unitary output for each service (e.g.,1410, 1415, 1420), and this information may be combined (e.g., as metadata describing the derived attributes of the various faces within the image) with a scaled image (generated by the image resizing service 1425) to be transmitted to the consuming entity (e.g., visualizer application 1430), among a variety of other examples. Although some of the examples have utilized image data and image processing, computer vision, and other image-related pipelines, it is indeed understood that such dynamic service replication within edge computing environments can be applied to a potentially limitless variety of other pipelines, services, applications, and data.

[0057] "Logic," as used herein, may refer to hardware, firmware, software, and / or combinations thereof to perform one or more functions. In various embodiments, logic may include a microprocessor or other processing element operable to execute software instructions, discrete logic such as an application-specific integrated circuit (ASIC), a programmed logic device such as a field-programmable gate array (FPGA), a memory device containing instructions, combinations of logic devices (such as would be found on a printed circuit board), or other suitable hardware and / or software. Logic may include one or more gates or other circuit components. In some embodiments, logic may also be implemented entirely in software.

[0058] A design can go through several stages, from generation to simulation to manufacturing. Data representing a design can represent the design in several ways. First, because it is useful for simulations, the hardware can be represented using a hardware description language (HDL) or another functional description language. Additionally, a circuit-level model with logic and / or transistor gates can be created at some stage of the design process. In addition, most designs eventually reach a stage of data representing the physical placement of various devices in the hardware model.In the case where conventional semiconductor manufacturing techniques are used, the data representing the hardware model may be the data indicating the presence or absence of various features on different mask layers for masks used to fabricate the integrated circuit. In some implementations, such data may be stored in a database file format such as Graphic Data System II (GDS II), Open Artwork System Interchange Standard (OASIS), or a similar format.

[0059] In some implementations, software-based hardware models, as well as HDL and other functional description language objects, may include, among other things, register transfer language (RTL) files. Such objects may be machine-parsed, allowing a design tool to accept the HDL object (or model), analyze the HDL object for attributes of the described hardware, and determine from the object a physical circuit and / or on-chip layout. The design tool's results may be used to manufacture the physical device. For example, a design tool may determine configurations of various hardware and / or firmware elements from the HDL object, such as bus widths, registers (including sizes and types), memory blocks, physical interconnect paths, fabric topologies, and other attributes implemented to realize the system modeled in the HDL object.Design tools may include tools for determining the topology and fabric configurations of System on Chip (SoC) and other hardware devices. In some cases, the HDL object can be used as a basis for developing models and design files that can be used by manufacturing facilities to produce the described hardware. Indeed, an HDL object itself can be provided as input to the manufacturing system's software to generate the described hardware.

[0060] In any representation of the design, the data may be stored in any form of machine-readable medium. A storage device or a magnetic or optical storage medium, such as a disc, may be the machine-readable medium for storing information transmitted via optical or electrical waves that are modulated or otherwise generated to transmit such information. When an electrical carrier wave indicating or carrying the code or design is transmitted, to the extent that copying, buffering, or retransmission of the electrical signal is performed, a new copy is made. Therefore, a communications provider or network operator may store on a tangible, machine-readable medium, at least temporarily, an item, such as information encoded in a carrier wave, embodying techniques of embodiments of the present disclosure.

[0061] A module, as used herein, refers to any combination of hardware, software, and / or firmware. As one example, a module includes hardware, such as a microcontroller, coupled to a non-volatile storage medium to store code adapted to be executed by the microcontroller. Therefore, in one embodiment, reference to a module refers to the hardware specifically configured to recognize and / or execute the code to be held on a non-volatile storage medium. Furthermore, in another embodiment, use of a module refers to the non-volatile storage medium containing code specifically adapted to be executed by the microcontroller to perform predetermined operations.And as can be seen, in yet another embodiment, the term module (in this example) may refer to the combination of the microcontroller and the non-volatile storage device. Often, module boundaries depicted as separate may generally vary and potentially overlap. For example, a first and a second module may share hardware, software, firmware, or a combination thereof, while potentially maintaining independent portions of hardware, software, or firmware. In one embodiment, the use of the term logic includes hardware such as transistors, registers, or other hardware such as programmable logic devices.

[0062] The use of the phrase 'to' or 'configured to' in one embodiment refers to arranging, assembling, manufacturing, offering for sale, importing, and / or designing a device, hardware, logic, or element to perform a designated or particular task. In this example, a device or element thereof that is not operating is still 'configured to' perform a designated task if it is designed, coupled, and / or connected to perform the designated task. As a purely illustrative example, a logic gate may provide a 0 or a 1 during operation. However, a logic gate that is 'configured to' provide an enable signal to a clock does not include every potential logic gate that can provide a 1 or a 0.Instead, the logic gate is one that is coupled in such a way that, during operation, the 1 or 0 output is intended to activate the clock. It should be noted again that the use of the term "configured to" does not require operation, but instead refers to the latent state of a device, hardware, and / or element, where the device, hardware, and / or element in the latent state is designed to perform a specific task when the device, hardware, and / or element is operating.

[0063] Furthermore, in one embodiment, use of the phrases "capable of" and / or "operative to" refers to a certain device, logic, hardware, and / or element configured in a manner to enable use of the device, logic, hardware, and / or element in a specified manner. As above, note that in one embodiment, use of "to," "capable of," or "operative to" refers to the latent state of a device, logic, hardware, and / or element when the device, logic, hardware, and / or element is not operational but configured in such a manner to enable use of a device in a specified manner.

[0064] A value, as used herein, includes any known representation of a number, a state, a logical state, or a binary logical state. Often, the use of logic levels, logic values, or logical values is also referred to as 1s and 0s, which simply represent binary logical states. For example, 1 refers to a high logic level, and 0 refers to a low logic level. In one embodiment, a memory cell, such as a transistor or flash cell, may be capable of holding a single logical value or multiple logical values. However, other representations of values have been used in computer systems. For example, the decimal number ten can also be represented as a binary value of 418A0 and as a hexadecimal letter A. Therefore, a value includes any representation of information that can be held in a computer system.

[0065] Furthermore, states can be represented by values and portions of values. As an example, a first value, such as a logical one, can represent a default or initial state, while a second value, such as a logical zero, can represent a non-default state. Additionally, in one embodiment, the terms "reset" and "set" refer to a default or updated value or state, respectively. For example, a default value potentially includes a high logical value, such as reset, while an updated value potentially includes a low logical value, such as set. It should be noted that any combination of values can be used to represent any number of states.

[0066] The above-discussed embodiments of methods, hardware, software, firmware, or code may be implemented via instructions or code stored on a machine-accessible, machine-readable, computer-accessible, or computer-readable medium and executable by a processing element. A non-transitory machine-accessible / readable medium includes any mechanism that provides (i.e., stores and / or transmits) information in a form readable by a machine, such as a computer or electronic system.A non-volatile machine-accessible medium includes, for example, random access memory (RAM), such as static RAM (SRAM) or dynamic RAM (DRAM); ROM; magnetic or optical storage media; flash memory devices; electrical storage devices; optical storage devices; acoustic storage devices; other forms of storage devices for holding information received from transient (propagated) signals (e.g., carrier waves, infrared signals, digital signals), etc., which must be distinguished from non-volatile media that can receive information therefrom.

[0067] Instructions used to program logic to perform embodiments of the disclosure may be stored within a memory in the system, such as DRAM, cache, flash memory, or other memory. Furthermore, the instructions may be distributed over a network or by other computer-readable media. Therefore, a machine-readable medium may include any mechanism for storing or transmitting information in a format readable by a machine (e.g., a computer).a computer), readable form include, but is not limited to, floppy disks, optical disks, compact disc read-only memory (CD-ROMs), and magneto-optical disk read-only memory (ROMs), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic or optical cards, flash memory, or any tangible machine-readable storage used in the transmission of information over the Internet via electrical, optical, acoustic, or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.).Accordingly, the computer-readable medium includes any type of tangible machine-readable medium suitable for storing or transmitting electronic instructions or information in a form readable by a machine (e.g., a computer).

[0068] The following examples relate to embodiments consistent with this specification. Example 1 is a non-transitory machine-readable storage medium having instructions stored thereon, the instructions executable to cause a machine to: receive data to be processed in an application, the application including a set of services, at least a particular one of the set of services to be executed on a first edge computing device, and the particular service to perform a set of operations on the data; analyze the data to determine characteristics of the data; determine an opportunity to replicate the particular service on multiple edge computing devices based on characteristics of the data;Determining that a second edge computing device is available to execute a replicated instance of the particular service, wherein the plurality of edge computing devices is to include the first edge computing device and the second edge computing device; and initiating replication of the particular service on the plurality of edge computing devices, wherein an output of an instance of the particular service executing on the first edge computing device and an output of the replicated instance of the particular service executing on the second edge computing device are to be combined to form a single output for the particular service.

[0069] Example 2 includes the subject matter of Example 1, wherein the opportunity to replicate the particular service is further determined based on attributes of the set of operations of the particular service.

[0070] Example 3 includes the subject matter of any of Examples 1-2, wherein the opportunity to replicate the particular service is further determined based on performance metrics defined for the application.

[0071] Example 4 includes the subject matter of any of Examples 1-3, wherein the determination of whether or not to replicate the particular service is dynamically determined based at least in part on the characteristics of the data.

[0072] Example 5 includes the subject matter of any of Examples 1-4, wherein the instructions are further executable to: create copies of the data based on the opportunity; and pass respective copies of the data to the plurality of edge computing devices to execute in instances of the particular service.

[0073] Example 6 includes the subject matter of any of Examples 1-5, wherein the instructions are further executable to: determine a plurality of segments of the data based on the characteristics of the data; and forward a respective segment of the data to the plurality of edge computing devices for execution in instances of the determined service.

[0074] Example 7 includes the subject matter of Example 6, wherein the opportunity is based on the set of operations being performed separately for each of the plurality of segments.

[0075] Example 8 includes the subject matter of Example 7, wherein the data includes image data, the set of operations includes image processing operations to be performed on a representation of a person or thing in the image data, wherein a plurality of different people or things in the image data are identified by analyzing the data, and the plurality of segments correspond to representations of the plurality of different people or things in the image data.

[0076] Example 9 includes the subject matter of any of Examples 1-8, wherein the instructions are further executable to: receive second data to be operated on by the set of operations in a second execution of the particular service; analyze the second data to determine characteristics of the second data; and determine that no opportunities exist to replicate the particular service in the second execution of the particular service based on the characteristics of the second data.

[0077] Example 10 includes the subject matter of any of Examples 1-9, wherein the data is analyzed using a machine learning algorithm to determine different persons, things, or entities described in the data, wherein the characteristics of the data correspond to the different persons, things, or entities described in the data.

[0078] Example 11 includes the subject matter of any of Examples 1-10, wherein the instructions are further executable to: generate the single output for the particular service from the output of the instance of the particular service executing on the first edge computing device and the output of the replicated instance of the particular service executing on the second edge computing device.

[0079] Example 12 includes the subject matter of Example 11, wherein an expected format is defined for the output of the particular service and the individual output is generated in the expected format.

[0080] Example 13 includes the subject matter of Example 12, wherein at least one of the output of the instance of the particular service executing on the first edge computing device or the output of the replicated instance of the particular service executing on the second edge computing device is not in the expected format.

[0081] Example 14 includes the subject matter of Example 12, wherein the single output is to be provided as an input to another service in the application or as an output of the application.

[0082] Example 15 is an apparatus including: an edge computing device including: a processor; memory; a network adapter for coupling to a network including a plurality of other edge computing devices; an orchestrator executable by the processor to: receive a particular workload in an application, the workload including one of a plurality of workloads to be executed in the application; and cause the particular workload to be executed by the edge computing device; and a service replication manager executable by the processor to: identify attributes of data to be processed in the particular workload; determine an opportunity to replicate the particular workload on one or more of the plurality of other edge computing devices based on the attributes of the data;Determining that a particular one of the plurality of other edge computing devices is available to execute a replicated instance of the particular workload; initiating replication of the particular workload on the particular other edge computing device; and generating a combined output for the particular workload based on an output from the edge computing device's execution of the particular workload and an output from the execution of the replicated instances of the particular workload.

[0083] Example 16 includes the subject matter of Example 15, wherein the service replication manager is to identify a plurality of segments of the data, a first of the plurality of segments is to be processed upon execution of the particular workload by the edge computing device, and a second of the plurality of segments is to be processed upon execution of the replicated instance of the particular workload by the particular other edge computing device.

[0084] Example 17 includes the subject matter of any of Examples 15-16, wherein the execution of the particular workload by the edge computing device and the execution of the replicated instance of the particular workload by the particular other edge computing device are at least partially parallel.

[0085] Example 18 includes the subject matter of any of Examples 15-17, wherein the opportunity to replicate the particular service is further determined based on attributes of the set of operations of the particular service.

[0086] Example 19 includes the subject matter of any of Examples 15-18, wherein the opportunity to replicate the particular service is further determined based on performance metrics defined for the application.

[0087] Example 20 includes the subject matter of any of Examples 15-19, wherein the determination of whether or not to replicate the particular service is dynamically determined based at least in part on the characteristics of the data.

[0088] Example 21 includes the subject matter of any of Examples 15-20, wherein the service replication manager is further executable to: create copies of the data based on the opportunity; and propagate respective copies of the data to the plurality of edge computing devices to execute in instances of the particular service.

[0089] Example 22 includes the subject matter of any of Examples 15-21, wherein the service replication manager is further operable to: receive second data to be operated on by the set of operations in a second execution of the particular service; analyze the second data to determine characteristics of the second data; and determine that no opportunities exist to replicate the particular service in the second execution of the particular service based on the characteristics of the second data.

[0090] Example 23 includes the subject matter of any of Examples 15-22, wherein the data is analyzed using a machine learning algorithm to determine different persons, things, or entities described in the data, wherein the characteristics of the data correspond to the different persons, things, or entities described in the data.

[0091] Example 24 includes the subject matter of any of Examples 15-23, wherein the service replication manager is further executable to: generate the single output for the particular service from the output of the instance of the particular service executing on the first edge computing device and the output of the replicated instance of the particular service executing on the second edge computing device.

[0092] Example 25 includes the subject matter of Example 24, wherein an expected format is defined for the output of the particular service and the individual output is generated in the expected format.

[0093] Example 26 includes the subject matter of Example 25, wherein at least one of the output of the instance of the particular service executing on the first edge computing device or the output of the replicated instance of the particular service executing on the second edge computing device is not in the expected format.

[0094] Example 27 includes the subject matter of Example 25, wherein the single output is to be provided as an input to another service in the application or as an output of the application.

[0095] Example 28 is a system including: a first edge computing device; a second edge computing device coupled to the first edge computing device in a network; and an edge orchestrator executable by a processor to: orchestrate execution of an application, wherein the application includes a chain of services; and cause a service in the chain of services to be executed by the first edge computing device; wherein the first edge computing device is configured to: receive data to be processed in the service; analyze the data to determine characteristics of the data; determine an opportunity to replicate the service to form multiple processing pipelines for the data based on the characteristics of the data; perform data scaling on the data based on the opportunity;Initiating replication of the service on the second edge computing device based on the data scaling; and generating a combined output for the service based on an output from the edge computing device executing the service and an output from the second edge computing device executing the replicated instances of the service.

[0096] Example 29 includes the subject matter of Example 28, wherein scaling the data includes replicating the data, segmenting the data, eliminating at least a portion of the data, or performing a data transformation on the data.

[0097] Example 30 includes the subject matter of any of Examples 28-29, wherein the second edge computing device is selected to execute a replicated instance of the service based on one or more of the availability of the second edge computing device, a geographic location of the second edge computing device, or hardware characteristics of the second edge computing device.

[0098] Example 31 includes the subject matter of any of Examples 28-30, wherein the opportunity to replicate the particular service is further determined based on attributes of the set of operations of the particular service.

[0099] Example 32 includes the subject matter of any of Examples 28-31, wherein the opportunity to replicate the particular service is further determined based on performance metrics defined for the application.

[0100] Example 33 includes the subject matter of any of Examples 28-32, wherein the determination of whether or not to replicate the particular service is dynamically determined based at least in part on the characteristics of the data.

[0101] Example 34 includes the subject matter of any of Examples 28-33, wherein the first edge computing device is further configured to: create copies of the data based on the opportunity; and pass respective copies of the data to the plurality of edge computing devices to execute in instances of the particular service.

[0102] Example 35 includes the subject matter of any of Examples 28-34, wherein the first edge computing device is further configured to: determine a plurality of segments of the data based on the characteristics of the data; and forward a respective segment of the data to the plurality of edge computing devices to execute in instances of the determined service.

[0103] Example 36 includes the subject matter of Example 35, wherein the opportunity is based on the set of operations being performed separately for each of the plurality of segments.

[0104] Example 37 includes the subject matter of Example 36, wherein the data includes image data, the set of operations includes image processing operations to be performed on a representation of a person or thing in the image data, wherein a plurality of different people or things in the image data are identified by analyzing the data, and the plurality of segments correspond to representations of the plurality of different people or things in the image data.

[0105] Example 38 includes the subject matter of any of Examples 28-37, wherein the first edge computing device is further configured to: receive second data to be operated on by the set of operations in a second execution of the particular service; analyze the second data to determine characteristics of the second data; and determine that no opportunities exist to replicate the particular service in the second execution of the particular service based on the characteristics of the second data.

[0106] Example 39 includes the subject matter of any of Examples 28-38, wherein the data is analyzed using a machine learning algorithm to determine different persons, things, or entities described in the data, wherein the characteristics of the data correspond to the different persons, things, or entities described in the data.

[0107] Example 40 includes the subject matter of any of Examples 28-39, wherein the first edge computing device is further configured to: generate the single output for the particular service from the output of the instance of the particular service executing on the first edge computing device and the output of the replicated instance of the particular service executing on the second edge computing device.

[0108] Example 41 includes the subject matter of Example 40, wherein an expected format is defined for the output of the particular service and the particular output is generated in the expected format.

[0109] Example 42 includes the subject matter of Example 41, wherein at least one of the output of the instance of the particular service executing on the first edge computing device or the output of the replicated instance of the particular service executing on the second edge computing device is not in the expected format.

[0110] Example 43 includes the subject matter of Example 41, wherein the single output is to be provided as an input to another service in the application or as an output of the application.

[0111] Example 44 is a method including: receiving data to be processed in an application, the application including a set of services, at least a particular one of the set of services to be executed on a first edge computing device, and the particular service to perform a set of operations on the data; analyzing the data to determine characteristics of the data; determining an opportunity to replicate the particular service on multiple edge computing devices based on characteristics of the data; determining that a second edge computing device is available to execute a replicated instance of the particular service, the plurality of edge computing devices to include the first edge computing device and the second edge computing device;and initiating a replication of the particular service on the plurality of edge computing devices, wherein an output of an instance of the particular service executing on the first edge computing device and an output of the replicated instance of the particular service executing on the second edge computing device are to be combined to form a single output for the particular service.;

[0112] Example 45 includes the subject matter of Example 44, wherein the opportunity to replicate the particular service is further determined based on attributes of the set of operations of the particular service.

[0113] Example 46 includes the subject matter of any of Examples 44-45, wherein the opportunity to replicate the particular service is further determined based on performance metrics defined for the application.

[0114] Example 47 includes the subject matter of any of Examples 44-46, wherein the determination of whether or not to replicate the particular service is dynamically determined based at least in part on the characteristics of the data.

[0115] Example 48 includes the subject matter of any of Examples 44-47, further including: generating copies of the data based on the opportunity; and passing respective copies of the data to the plurality of edge computing devices to execute in instances of the particular service.

[0116] Example 49 includes the subject matter of any of Examples 44-48, further including: determining a plurality of segments of the data based on the characteristics of the data; and forwarding a respective segment of the data to the plurality of edge computing devices to execute in instances of the determined service.

[0117] Example 50 includes the subject matter of Example 49, wherein the opportunity is based on the set of operations being performed separately for each of the plurality of segments.

[0118] Example 51 includes the subject matter of Example 50, wherein the data includes image data, the set of operations includes image processing operations to be performed on a representation of a person or thing in the image data, wherein a plurality of different people or things in the image data are identified by analyzing the data, and the plurality of segments correspond to representations of the plurality of different people or things in the image data.

[0119] Example 52 includes the subject matter of any of Examples 44-51, further including: receiving second data to be operated on by the set of operations in a second execution of the particular service; analyzing the second data to determine characteristics of the second data; and determining that no opportunities exist to replicate the particular service in the second execution of the particular service based on the characteristics of the second data.

[0120] Example 53 includes the subject matter of any of Examples 44-52, wherein the data is analyzed using a machine learning algorithm to determine different persons, things, or entities described in the data, wherein the characteristics of the data correspond to the different persons, things, or entities described in the data.

[0121] Example 54 includes the subject matter of any of Examples 44-53, further comprising: generating the single output for the particular service from the output of the instance of the particular service executing on the first edge computing device and the output of the replicated instance of the particular service executing on the second edge computing device.

[0122] Example 55 includes the subject matter of Example 54, wherein an expected format is defined for the output of the particular service and the particular output is produced in the expected format.

[0123] Example 56 includes the subject matter of Example 55, wherein at least one of the output of the instance of the particular service executing on the first edge computing device or the output of the replicated instance of the particular service executing on the second edge computing device is not in the expected format.

[0124] Example 57 includes the subject matter of Example 55, wherein the single output is to be provided as an input to another service in the application or as an output of the application.

[0125] Example 58 is a system having means for performing the method of any one of Examples 44-57.

[0126] Throughout this specification, reference to "one embodiment," or "an embodiment," means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Therefore, the phrase "in one embodiment" in various places throughout this specification does not necessarily refer to the same embodiment. Further, the particular features, structures, or characteristics may be combined in one or more embodiments in any suitable manner.

[0127] In the foregoing specification, a detailed description has been given with reference to specific exemplary embodiments. It will, however, be apparent that various modifications and changes may be made therein without departing from the broader spirit and scope of the disclosure as set forth in the appended claims. The specification and drawings are, accordingly, to be considered in an illustrative sense, not in a limiting sense. Furthermore, the foregoing use of embodiment and exemplary language do not necessarily refer to the same embodiment or example, but may refer to different and various embodiments, as well as potentially to the same embodiment.

Claims

[1] At least one non-transitory machine-readable storage medium having instructions stored thereon, the instructions being executable to cause a machine to: Receiving data to be processed in an application, the application including a set of services, at least one particular one of the set of services to be executed on a first edge computing device, and the particular service to perform a set of operations on the data; Analyzing the data to determine characteristics of the data; determining an opportunity to replicate the particular service on multiple edge computing devices based on characteristics of the data; Determining that a second edge computing device is available to execute a replicated instance of the particular service, wherein the plurality of edge computing devices is to include the first edge computing device and the second edge computing device; and Initiating a replication of the particular service on the plurality of edge computing devices, wherein an output of an instance of the particular service executing on the first edge computing device and an output of the replicated instance of the particular service executing on the second edge computing device are to be combined to form a single output for the particular service. [2] The storage medium of claim 1, wherein the opportunity to replicate the particular service is further determined based on attributes of the set of operations of the particular service. [3] The storage medium of claim 1, wherein the opportunity to replicate the particular service is further determined based on performance metrics defined for the application. [4] The storage medium of claim 1, wherein the determination of whether or not to replicate the particular service is dynamically determined based at least in part on the characteristics of the data. [5] The storage medium of claim 1, wherein the instructions are further executable to: Creating copies of the data based on the opportunity; and Passing respective copies of the data to the multiple edge computing devices for execution in instances of the particular service. [6] The storage medium of claim 1, wherein the instructions are further executable to: Determining multiple segments of the data based on the characteristics of the data; and Forwarding a respective segment of the data to the plurality of edge computing devices to execute in instances of the particular service. [7] The storage medium of claim 6, wherein the opportunity is based on the set of operations being performed separately for each of the plurality of segments. [8] The storage medium of claim 7, wherein the data includes image data, the set of operations includes image processing operations to be performed on a representation of a person or thing in the image data, wherein a plurality of different people or things in the image data are identified by analyzing the data, and the plurality of segments correspond to representations of the plurality of different people or things in the image data. [9] The storage medium of claim 1, wherein the instructions are further executable to: receiving second data to be processed by the set of operations in a second execution of the particular service; analyzing the second data to determine characteristics of the second data; and Determining that no opportunities exist to replicate the particular service in the second execution of the particular service based on the characteristics of the second data. [10] The storage medium of claim 1, wherein the data is analyzed using a machine learning algorithm to determine different persons, things, or entities described in the data, wherein the characteristics of the data correspond to the different persons, things, or entities described in the data. [11] The storage medium of claim 1, wherein the instructions are further executable to: Generating the single output for the particular service from the output of the instance of the particular service executing on the first edge computing device and the output of the replicated instance of the particular service executing on the second edge computing device. [12] The storage medium of claim 11, wherein an expected format is defined for the output of the particular service and the single output is generated in the expected format. [13] The storage medium of claim 12, wherein at least one of the output of the instance of the particular service executing on the first edge computing device or the output of the replicated instance of the particular service executing on the second edge computing device is not in the expected format. [14] The storage medium of claim 12, wherein the single output is to be provided as an input to another service in the application or as an output of the application. [15] Facility comprising: an edge computing device comprising: a processor; a memory; a network interface controller for coupling to a network comprising a plurality of other edge computing devices; an orchestrator executable by the processor to: Receiving a particular workload in an application, wherein the workload comprises one of a plurality of workloads to be executed in the application; and Causing the particular workload to be executed by the edge computing device; and a service replication manager executable by the processor to: Identifying attributes of data to be processed in the specific workload; determining an opportunity to replicate the particular workload on one or more of the plurality of other edge computing devices based on the attributes of the data; determining that a particular one of the plurality of other edge computing devices is available to execute a replicated instance of the particular workload; Initiating replication of the particular workload on the particular other edge computing device; and Generating a combined output for the particular workload based on an output from the execution of the particular workload by the edge computing device and an output from the execution of the replicated instances of the particular workload. [16] The apparatus of claim 15, wherein the service replication manager is to identify a plurality of segments of the data, a first of the plurality of segments is to be processed upon execution of the particular workload by the edge computing device, and a second of the plurality of segments is to be processed upon execution of the replicated instance of the particular workload by the particular other edge computing device. [17] The apparatus of claim 15, wherein the execution of the particular workload by the edge computing device and the execution of the replicated instance of the particular workload by the particular other edge computing device are at least partially parallel. [18] System comprising: a first edge computing device; a second edge computing device coupled to the first edge computing device in a network; and an edge orchestrator, executable by a processor to: Orchestrating the execution of an application, the application comprising a chain of services; and causing a service in the chain of services to be executed by the first edge computing device; wherein the first edge computing device is configured to: Receiving data to be processed in the service; Analyzing the data to determine characteristics of the data; Determining an opportunity to replicate the service to form multiple processing pipelines for the data based on the characteristics of the data; Performing data scaling on the data based on the opportunity; Initiating replication of the service on the second edge computing device based on the data scaling; and Generating a combined output for the service based on an output from the execution of the service by the edge computing device and an output from the execution of the replicated instances of the service by the second edge computing device. [19] The system of claim 18, wherein scaling the data includes replicating the data, segmenting the data, eliminating at least a portion of the data, or performing a data transformation on the data. [20] The system of claim 18, wherein the second edge computing device is selected to execute a replicated instance of the service based on one or more of the availability of the second edge computing device, a geographic location of the second edge computing device, or hardware characteristics of the second edge computing device.

Images