Serverless Capacity Planning Using API-Level Workload Characterization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cost optimization techniques for serverless instances ignore memory configuration, leading to suboptimal resource allocation and inefficient deployment of applications, as they rely on synthetic benchmarks that fail to capture actual application requirements.
Innovation Solution
A workload characterization-based method that refactors applications into independent APIs, uses an Elastic File System for data communication, and optimizes compute requirements by varying core allocation to determine the optimal number of cores for each API, enabling cost-effective and high-performance execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If serverless platforms provision more resources per instance, then AI workloads can be deployed, but deployment cost increases
Solution Approach 1:
The patent dynamically adjusts resource parameters (CPU cores, memory, storage) based on actual workload characteristics. By measuring execution time, throughput, and resource utilization metrics from synthetic benchmarks, the system determines optimal resource configurations that balance capability and cost, avoiding over-provisioning while maintaining AI workload performance
Solution Approach 2:
The patent implements dynamic resource allocation where serverless instance configurations are adjusted based on real-time workload demands. The system continuously monitors performance metrics and adapts resource provisioning levels, transitioning from static over-provisioning to dynamic optimization that maintains AI workload capability while reducing unnecessary resource consumption
2Productivity
If memory size of serverless instances is increased, then AI workload performance improves, but deployment cost increases
Solution Approach 1:
The patent systematically varies memory size parameters in synthetic benchmarks to identify the optimal memory configuration for AI workloads. By measuring performance metrics across different memory allocations, the system determines the minimum memory size required to achieve acceptable AI workload performance, avoiding excessive memory provisioning that would increase deployment costs
3Extent of automation
If synthetic benchmarks are used for resource prediction, then resource allocation can be automated, but actual application requirements are not captured
Solution Approach 1:
The patent performs preliminary resource characterization using synthetic benchmarks before actual application deployment. By pre-measuring resource requirements under controlled synthetic workloads, the system builds a resource profile that guides initial resource allocation decisions, enabling automated resource provisioning while accounting for application-specific characteristics
Solution Approach 2:
The patent implements feedback mechanisms where actual application performance metrics are continuously monitored and fed back into the resource prediction model. This feedback loop refines the synthetic benchmark predictions by incorporating real-world application behavior data, improving the accuracy of resource requirement predictions over time
Data Source
AI summary
Recent techniques for workload characterization of an application to be executed in a serverless execution environment or cloud are based on benchmark-approximation. Multiple microbenchmarks are run against the multiple VM configurations and a score is calculated which is used for mapping futuristic workloads to the appropriate configuration. Embodiments herein disclose method and system for workload characterization-based capacity planning of an actual application running on-premise with different configurations of the same machine and providing a cost-effective and high-performance serverless execution environment. Resource demand of each API in the application workflow is evaluated. Based on the resource demand of each API, a mapping is performed to the serverless platform on cloud. Additionally, characterization of threads within each API is performed and each thread is mapped to a serverless instance based on its resource requirements.