Storage Job Workflow Using Dynamic Task Queues and Containers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage service job workflows require different static workflows for various types of storage service jobs, which can be cumbersome and inefficient, especially when handling multiple entities, and often impact system performance.
Innovation Solution
A dynamic workflow system that allows storage service jobs to be performed without predefined sequences, utilizing virtualization containers to dynamically determine task orders and resource allocation based on real-time conditions, with checkpointing and heartbeat signals for resilience and resource management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If different static workflows are specified for various types of storage service jobs, then job execution can be controlled, but system complexity and operational burden increase
Solution Approach 1:
The patent implements dynamic workflow generation where the system automatically determines the sequence of tasks based on real-time conditions and job requirements, rather than using predefined static workflows. The workflow manager dynamically selects and orders tasks from available templates, allowing flexible adaptation to different storage service job types without manual configuration of each workflow variant.
Solution Approach 2:
The patent creates a universal workflow management system that handles multiple types of storage service jobs (backup, restore, replication, migration) through a single dynamic workflow engine. Instead of maintaining separate static workflows for each job type, the system uses a unified approach with task templates and dynamic sequencing that works across all job types.
2Productivity
If static workflows are used for storage service jobs, then execution sequence is defined, but resource utilization efficiency decreases
Solution Approach 1:
The system dynamically determines task execution sequences based on real-time resource availability, job priorities, and dependencies. This allows optimal resource allocation and execution ordering that adapts to changing system conditions, improving resource utilization efficiency compared to fixed static workflows that cannot respond to real-time conditions.
Solution Approach 2:
The workflow manager autonomously determines task sequencing and resource allocation without external intervention. The system self-adjusts the execution plan based on job requirements and system state, eliminating the need for manual workflow design and enabling more efficient resource utilization through automated optimization.
3Adaptability or versatility
If multiple static workflows are maintained for different entities, then specific job requirements can be met, but system complexity increases
Solution Approach 1:
The patent segments storage service jobs into standardized task templates with specific functions (e.g., backup task, restore task, replication task). Each template can be independently configured and combined dynamically to handle different job types and entity requirements, reducing the need for maintaining separate complete workflows for each scenario.
Solution Approach 2:
The system dynamically assembles task sequences from available templates based on job requirements, allowing flexible adaptation to different entities and job types. This dynamic composition replaces the need for maintaining multiple static workflows, as the same template library can be combined in different ways to meet various requirements.
Data Source
AI summary
An indication of a storage service job to be performed is received. A task to be performed for the storage service job is determined. The task is added to a work queue. Execution of one or more tasks in the work queue that includes the task is dynamically managed. Resources are dynamically allocated to one or more virtualization containers that are assigned to execute the one or more tasks in the work queue. An identification of one or more new tasks to be performed for the storage service job is received from one of the virtualization containers executing the task. The one or more new tasks are added to the work queue.