Storage Job Workflow Using Dynamic Task Queues and Containers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage service providers face inefficiencies in managing diverse storage service jobs due to the need for specifying different static workflows for various entities, leading to complexity and potential performance impacts.
Innovation Solution
A dynamic workflow system that allows storage service jobs to be performed without predefined sequences, utilizing virtualization containers to dynamically determine task orders and resource allocation based on real-time conditions, with checkpointing and heartbeat signals for resilience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If static workflows are specified for different storage service jobs, then job execution can be defined, but system complexity increases and performance may be impacted
Solution Approach 1:
The patent implements dynamic workflow execution by allowing tasks to be added, removed, or reordered during job execution based on real-time conditions. The workflow is no longer static but adapts to changing requirements, enabling flexible resource allocation and task scheduling without predefined rigid sequences.
Solution Approach 2:
The system enables self-service workflow management where the storage service job can autonomously determine task sequencing and resource allocation based on its own state and external conditions. The workflow manager dynamically adjusts execution plans without requiring manual reconfiguration for each job type.
2Adaptability or versatility
If multiple static workflows are maintained for different entities, then specific job requirements can be met, but management becomes cumbersome
Solution Approach 1:
The patent creates a universal workflow management framework that can handle multiple entity types and job requirements through a single dynamic system. Instead of maintaining separate static workflows for different entities, the system uses a unified approach that adapts to any entity's specific needs through dynamic task configuration and resource allocation.
3Productivity
If predefined task sequences are used, then execution order is controlled, but flexibility to handle varying job types is reduced
Solution Approach 1:
The system dynamically determines task execution sequences based on real-time conditions, data availability, and job requirements. Tasks can be scheduled, paused, or reordered during execution without requiring predefined sequences, enabling the system to optimize productivity for each specific job type while maintaining flexibility.
Data Source
AI summary
An indication of a storage service job to be performed is received. A task to be performed for the storage service job is determined. The task is added to a work queue. Execution of one or more tasks in the work queue that includes the task is dynamically managed. Resources are dynamically allocated to one or more virtualization containers that are assigned to execute the one or more tasks in the work queue. An identification of one or more new tasks to be performed for the storage service job is received from one of the virtualization containers executing the task. The one or more new tasks are added to the work queue.