Cross-Cloud Workflow Management via Stream Matrix
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud computing systems lack effective management techniques for performance-centric workflows that span multiple clouds and cloud technologies, particularly in terms of resource allocation and workflow anatomy, leading to inefficiencies in virtual workload management.
Innovation Solution
A method for workflow management in cloud computing systems involving the generation of a workflow graph, stream matrix, and processing to place work-elements in a platform service pipeline based on resource availability, with automatic extraction of parallel workstreams and hierarchical workflow scheduling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If workflows span multiple clouds and cloud technologies, then workflow versatility and resource access improve, but management complexity and coordination difficulty increase
Solution Approach 1:
The patent segments workflows into work-elements and organizes them in a workflow graph with nodes and edges. Each work-element represents a discrete unit that can be independently managed and executed across different cloud platforms, reducing the complexity of managing multi-cloud workflows as a monolithic entity.
Solution Approach 2:
The patent creates a universal workflow management system that can handle workflows across multiple cloud platforms (AWS, Azure, GCP, OpenStack) and technologies (VMs, containers) through a common framework. The workflow graph and stream matrix provide a platform-agnostic representation that enables portable workflow execution.
2Reliability
If performance-centric workflows require dedicated resources, then workflow execution reliability improves, but resource allocation complexity and system overhead increase
Solution Approach 1:
The patent performs preliminary validation of the workflow graph to ensure proper structure and resource requirements before execution. The system pre-processes the workflow definition, validates node connections, and prepares resource allocation plans in advance, ensuring reliable execution without requiring complex real-time resource management.
Solution Approach 2:
The patent implements a feedback mechanism where the system monitors workflow execution and resource availability, then dynamically adjusts resource allocation. The workflow manager receives feedback from cloud platforms about resource status and uses this information to optimize resource distribution while maintaining execution reliability.
3Productivity
If workflows are processed sequentially, then resource allocation simplicity improves, but execution time and productivity decrease
Solution Approach 1:
The patent segments workflows into independent work-elements that can be executed in parallel. The workflow graph structure identifies which work-elements have no dependencies on others, allowing the system to spawn multiple parallel execution streams without requiring complex coordination between all workflow components.
Solution Approach 2:
The patent transforms the sequential workflow representation into a multi-dimensional structure using the stream matrix, which organizes work-elements by execution stream rather than sequential order. This dimensional transformation enables parallel processing while maintaining workflow integrity through the matrix structure that tracks dependencies across streams.
4Adaptability or versatility
If cloud resource availability varies dynamically, then system adaptability improves, but workflow scheduling complexity and decision-making overhead increase
Solution Approach 1:
The patent implements a scheduling approach that places work-elements in the platform service pipeline based on current resource availability without requiring perfect optimization. The system takes partial action by scheduling work-elements when resources are available rather than waiting for optimal conditions, reducing scheduling complexity while maintaining good resource utilization.
Solution Approach 2:
The workflow manager autonomously monitors cloud resource availability and automatically adjusts workflow scheduling decisions without requiring external intervention or complex centralized coordination. Each cloud platform reports its own resource status, and the workflow manager uses this self-reported information to make scheduling decisions, reducing the overall system's decision-making overhead.
Data Source
AI summary
A method of workflow management in a cloud computing system that includes generating a workflow graph from a workflow definition, the workflow graph including nodes representing work-elements; generating a stream matrix from the workflow graph, the stream matrix including pointers to lists of the work-elements, each of the lists representing a workstream; processing the stream matrix to place work-elements in a platform service pipeline for the cloud computing system based on resource availability of the platform service pipeline; and removing work-elements from the lists and the platform service pipeline upon completion.