Serverless Workflow Resource Backfilling During Task Sleep States
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current serverless workflow cloud environments fail to deploy additional processes when insufficient worker node resources are available, as they only allocate resources when a process finishes its life-cycle and do not efficiently utilize resources during sleep states of long-running processes.
Innovation Solution
Implementing an intelligent process management system that analyzes historical data to schedule short-running processes during the sleep states of long-running processes, using a process analyzer, predictor, backfill handler, and timeout handler to optimize resource utilization without disrupting the long-running processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If resources are allocated only when a process finishes its life-cycle, then resource management is simple, but resource utilization during sleep states is wasted
Solution Approach 1:
The system performs preliminary actions by allocating resources in advance during sleep states before the process needs them again. The scheduler identifies upcoming wake times and pre-allocates worker nodes during the sleep period, so resources are ready when needed without requiring continuous allocation.
Solution Approach 2:
The resource allocation system transitions from a static model (allocate only when needed) to a dynamic model that continuously monitors process states, predicts wake times, and adjusts resource allocation in real-time based on whether processes are in sleep or active states.
2Reliability
If additional worker nodes are provisioned to handle process deployment, then process deployment is ensured, but infrastructure cost increases
Solution Approach 1:
The system merges the resource pools of multiple processes by allowing them to share worker nodes during their respective sleep states. Instead of dedicating exclusive worker nodes to each process, the scheduler combines available capacity across processes to serve deployment requests using existing infrastructure.
Solution Approach 2:
The system changes the operational parameters of worker nodes by transitioning them between active and sleep states, and by dynamically adjusting which processes are assigned to which nodes based on real-time resource availability and process state, rather than maintaining fixed assignments.
3Use of energy by moving object
If processes are deleted when entering sleep state, then resource consumption is reduced, but process resumption capability is lost
Solution Approach 1:
The system creates a lightweight representation or copy of the process state during sleep rather than deleting the process entirely. This allows the process to be quickly restored or resumed when needed without maintaining full resource allocation, balancing resource savings with resumption capability.
Data Source
AI summary
Intelligent process management is provided. A start time is determined for an additional process to be run on a worker node within a duration of a sleep state of a task of a process already running on the worker node by adding a first defined buffer time to a determined start time of the sleep state of the task. A backfill time is determined for the additional process by subtracting a second defined buffer time from a determined end time of the sleep state of the task. A scheduling plan is generated for the additional process based on the start time and the backfill time corresponding to the additional process. The scheduling plan is executed to run the additional process on the worker node according to the start time and the backfill time corresponding to the additional process.