Workflow Graph Coordination for Asynchronous Multimedia Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing edge computing systems face challenges in efficiently managing computational workflows with varying dependencies and scaling requirements, necessitating frequent software reconfiguration to adapt to changing data conditions without incurring downtime.
Innovation Solution
A distributed asynchronous dependency-based content processing framework that utilizes a workflow manager to generate and manage task messages across multiple processing nodes, leveraging advanced message queuing protocols to handle data dependencies and enable scalable, configurable, and extensible workflows.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If distributed computing is implemented at network edge to improve response times and save bandwidth, then network efficiency is improved, but system complexity and difficulty of workflow management increase
Solution Approach 1:
The system segments workflows into discrete, independently manageable tasks that can be distributed across multiple edge computing nodes. Each task is defined with specific inputs, outputs, and dependencies, allowing the workflow to be divided and executed in parallel across the distributed network while maintaining overall coordination through the workflow graph structure.
Solution Approach 2:
The patent introduces a workflow graph as an intermediary data structure that mediates between the centralized workflow definition and distributed task execution. The workflow graph contains task definitions, data dependencies, and execution metadata that can be broadcast to and interpreted by multiple edge nodes, simplifying the management of distributed computing complexity.
2Adaptability or versatility
If workflows are reconfigured to adapt to changing data conditions, then system adaptability is improved, but software development costs and downtime increase
Solution Approach 1:
The system implements dynamic workflow reconfiguration where the workflow graph can be modified at runtime without stopping the entire system. Tasks can be added, removed, or reconfigured based on changing data conditions, and the workflow manager dynamically updates the task definitions and dependencies, allowing continuous operation during adaptation.
Solution Approach 2:
The patent employs preliminary configuration where workflow templates and task definitions are pre-defined with all possible dependencies and parameters. When reconfiguration is needed, the system activates pre-prepared alternative task configurations from the workflow graph rather than creating new tasks from scratch, reducing the time and development cost of adaptation.
3Productivity
If computation requirements scale with data volume in distributed environment, then processing capacity is improved, but coordination overhead and message passing complexity increase
Solution Approach 1:
The workflow graph structure serves multiple functions simultaneously: it defines task logic, specifies data dependencies, guides task distribution across nodes, and coordinates result aggregation. This universal data structure reduces coordination overhead by consolidating multiple control functions into a single standardized format that all nodes can interpret.
Solution Approach 2:
The system implements feedback mechanisms where task results are broadcast back to the workflow manager and other nodes through standardized message passing. The workflow manager uses this feedback to track execution progress, manage dependencies, and trigger subsequent tasks, enabling automatic coordination that scales with processing capacity without proportionally increasing overhead.
Data Source
AI summary
Method for distributed asynchronous dependency-based content processing includes: receiving a multi-media digital content and a task to be performed on the multi-media digital content; determining types of the multi-media digital content; generating a workflow graph for each type of the content, each workflow graph including data dependency conditions of the task; generating a task message for each workflow graph; broadcasting the task messages to a respective task queue of an initial state of a workflow manager; responding to the broadcast task messages by a respective processing node; processing the task based on the respective workflow graph in the broadcast task message, and including a result of the processing for each workflow graph in a result message; broadcasting the result messages by the respective processing node; accumulating the broadcast result messages responsive to respective workflow graphs; and outputting the accumulated results to a user or an external system.