Dynamic Workflow Module Distribution Across Compute Grids
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing workflow processing technologies face challenges in optimizing workflows across distributed computing systems with limited and variable network links and rapid changes in resource availability, particularly in big-data systems like remote sensing, where data needs to be transmitted and processed efficiently across distant devices.
Innovation Solution
A method and apparatus for dynamically distributing, deploying, and configuring optimizable code modules using a content catalog with metadata, allowing modules to be statically or dynamically optimized and updated by a workflow manager, enabling adaptive resource utilization and efficient data processing across a grid of compute devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If static deployment of code is used, then reliability is improved, but adaptability deteriorates
Solution Approach 1:
The patent applies dynamics by transitioning from static code deployment to dynamic module loading and unloading. The system can load code modules on-demand based on current resource availability and workflow requirements, allowing the computing system to adapt its functionality dynamically rather than requiring complete software rewrites for new capabilities.
Solution Approach 2:
The patent segments code into independent, reusable modules that can be selectively loaded and unloaded. This modular approach allows the system to maintain stability through reliable module interfaces while achieving adaptability by composing different module combinations based on current needs, resolving the contradiction between reliability and adaptability.
2Adaptability or versatility
If complete software rewrites are performed, then adaptability is improved, but loss of time increases
Solution Approach 1:
By dividing software into independent modules, the system can add or update individual modules without rewriting the entire software system. This modular segmentation enables rapid adaptation to new requirements through selective module loading, dramatically reducing the time loss associated with complete software rewrites.
Solution Approach 2:
The system pre-compiles and stores code modules in a repository, preparing them in advance for deployment. When new processing capabilities are needed, the system can quickly load pre-prepared modules rather than compiling from source code, significantly reducing the time required to adapt to changing requirements.
3Ease of operation
If centralized workflow management is used, then ease of operation is improved, but device complexity increases
Solution Approach 1:
The patent introduces a workflow manager as an intermediary component that coordinates between distributed computing devices. This mediator handles the complexity of resource allocation, module loading, and workflow coordination centrally, while presenting a simplified interface to individual devices, thus maintaining ease of operation without increasing device-level complexity.
Solution Approach 2:
The workflow manager implements universal functionality that can handle multiple types of computing devices, data types, and workflow patterns through a single unified system. This multi-functional approach simplifies operation by providing a single point of coordination while managing the inherent complexity of distributed systems through standardized protocols and interfaces.
4Adaptability or versatility
If data is transmitted over network links, then adaptability is improved, but loss of energy increases
Solution Approach 1:
The system performs preliminary actions by loading and caching code modules locally before they are needed for data processing. This pre-loading of processing capabilities reduces the frequency and amount of data that needs to be transmitted over network links, thereby reducing energy consumption while maintaining adaptability through on-demand module loading.
Solution Approach 2:
The patent implements local quality by enabling code modules to be cached and executed locally on computing devices rather than requiring continuous network access. This allows data processing to occur locally with minimal energy consumption, while the system maintains adaptability by can loading additional modules from the network only when specifically needed, rather than continuously.
Data Source
AI summary
The system provides a method and apparatus for the dynamic distribution, deployment, and configuration of optimizable code modules for use with software workflows running on a single compute device or across a network connected grid of compute devices. The system comprises one or more collections of software and data modules stored in a content catalog, conforming to a defined interface, and having metadata conforming to a schema that enables the modules to be statically or dynamically optimized by the controlling workflow and a workflow manager. The system provides a service that enables code modules to be located, deployed, configured, and updated by the controlling workflow, the workflow manager, or a remote manager.