Shared Container Table Mitigates Serverless Cold Start Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Serverless platforms face performance penalties due to 'cold start' issues, where the time to instantiate a new container increases response times and leads to inefficient resource usage, especially with frequent short-lived function executions.
Innovation Solution
Implementing a container table that shares information about available containers across multiple serverless platforms, allowing requests to be routed to platforms with existing compatible containers, thereby avoiding the need for cold starts and reducing response times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a new container is instantiated to handle a request, then the serverless platform can service the request, but the response time increases due to container initialization
Solution Approach 1:
The system pre-initializes containers and maintains them in a ready state before requests arrive. The container manager continuously monitors and sustains containers in memory, so when a request comes in, the container is already prepared and can execute immediately without cold start overhead.
Solution Approach 2:
The system creates copies of container images and sustains multiple instances in memory. When a request arrives, the system can quickly allocate an existing copied container instance rather than creating a new one from scratch, significantly reducing initialization time.
2Loss of time
If containers are sustained in memory to avoid cold starts, then response time improves, but resource consumption increases
Solution Approach 1:
The container management system dynamically adjusts the number and state of sustained containers based on real-time demand. Containers are sustained in a warm state when needed and can be transitioned to or from dormant states, allowing the system to optimize between response time and resource usage by adapting container lifecycle to actual workload patterns.
Solution Approach 2:
The system changes the operational parameters of containers by sustaining them in different states (active, dormant, suspended) with varying resource allocations. This allows flexible control over resource consumption while maintaining the ability to quickly service requests by promoting dormant containers to active state when needed.
3Adaptability or versatility
If multiple serverless platforms operate independently, then each platform maintains its own containers, but cold starts occur when no suitable container is available
Solution Approach 1:
The system creates a universal container pool that can service requests from multiple serverless platforms. Containers are sustained in a shared environment and can be allocated to any platform that needs them, making the container infrastructure multi-functional and platform-agnostic, thereby reducing cold starts across all participating platforms.
Solution Approach 2:
The system introduces a container manager as an intermediary layer between multiple serverless platforms and the underlying container infrastructure. This mediator sustains and manages containers in a shared pool, allowing platforms to borrow containers when needed without experiencing cold starts, while maintaining their own operational independence.
Data Source
AI summary
Shared function execution mitigates cold start penalties for execution units. When a serverless platform receives a request, the request is performed when the serverless platform has a warm execution unit for the request. If a warm execution unit is not available or running, the serverless platform may send the request to another serverless platform rather than cold start an execution unit. The cold start is performed when the warm execution unit is not available at other platforms.