Execution Graphs for Zero-Downtime Edge Data Processing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Developers face issues with application updates in edge computing environments, including application downtime, latency, scalability challenges, managing diverse data sources and destinations, and vulnerabilities from reloading user-defined dynamically linked libraries.

Innovation Solution

Implementing a programmable connector that constructs an execution graph for data processing using modules in a portable binary code format, allowing updates to target specific modules without recompiling the entire application, and using an update monitor to manage module changes without interrupting data processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If the application is updated by modifying a module, then the application functionality is improved, but the underlying infrastructure must reboot causing application downtime and latency

Engineering Contradiction:

Improveapplication update capabilityVSAvoidapplication downtime and latency

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The application is divided into independent modules that can be updated separately. Each module is containerized and can be loaded independently, allowing specific modules to be updated without requiring a full application reboot. This segmentation enables hot-swapping of modules while the application continues to run.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A module loader acts as an intermediary between the updated modules and the running application. The loader receives updated modules, validates them, and loads them into the application's memory space without requiring a reboot. This intermediary enables seamless module updates while maintaining application continuity.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If the entire application is recompiled for module modification, then the module update is achieved, but the complexity of the update process increases

Engineering Contradiction:

Improvemodule update capabilityVSAvoidupdate process complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The application compilation process is segmented so that only individual modules need to be compiled, not the entire application. Each module can be independently compiled, tested, and packaged, significantly reducing the complexity of the update process compared to full application recompilation.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system changes from a monolithic compilation model to a modular compilation model. By changing the compilation parameter from 'entire application' to 'individual module', the complexity is reduced while maintaining the ability to update functionality.

Inventive Principle:

Principle #35Parameter changes

3Adaptability or versatility

If user-defined dynamically linked libraries are reloaded during updates, then the updated functionality is loaded, but security vulnerabilities are introduced

Engineering Contradiction:

Improvedynamic library updatingVSAvoidapplication security

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

A secure module loader acts as an intermediary that validates and verifies updated modules before loading them into the application. The loader checks digital signatures, validates module integrity, and ensures security policies are met before allowing the module to be loaded, preventing security vulnerabilities from being introduced during updates.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

Security validation and verification are performed as preliminary actions before the actual module loading occurs. The module loader checks for security issues, validates signatures, and verifies integrity beforehand, ensuring that only secure modules are loaded into the running application.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentEP4625153A1Execution graphs for data processing agility in an edge computing environment

Publication Date: 2025.10.01 MICROSOFT TECHNOLOGY LICENSING LLC

Figure 1
Figure 2
Figure 3

AI summary

The techniques described herein improve the management and execution of data processing in an edge computing environment using a programmable connector. The programmable connector defines a plurality of data sources, one or more data destinations, a plurality of operators, and a plurality of edges that are collectively useable to construct an execution graph. The execution graph provides a structure for the processing of data on a cluster of host servers in the edge computing environment. Stated alternatively, the execution graph models the flow of data from input data sources that are connected to the edge computing environment to output data destinations that are also connected to the edge computing environment. Accordingly, a node in the execution graph represents an operator and an edge in the execution graph connects two nodes.