Serverless Container Multiplexing for Cold-Start and Memory Overhead
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current serverless computing environments suffer from inefficiencies such as low resource utilization, long function response latencies, and overheads due to virtualization layers, cold starts, RPC/HTTP invocations, and global storage accesses, leading to increased response times and idle times from synchronous wait patterns.
Innovation Solution
Implement a dispatcher in a container that manages function execution by creating groups of handlers to process requests efficiently, utilizing shared memory and file resources, and employing smart handler scheduling to optimize resource usage and reduce redundant memory replication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple functions are executed in separate containers with virtualization, then each function gets isolated execution environment, but resource utilization decreases and overhead increases
Solution Approach 1:
The patent merges multiple function executions into a single container by creating multiple handler processes that share the same execution environment. This eliminates redundant virtualization overhead while maintaining isolation through process-level separation, thereby improving resource utilization without sacrificing execution reliability.
Solution Approach 2:
The container is designed to handle multiple functions simultaneously through a universal execution mechanism. The dispatcher can allocate different handler processes to different functions within the same container, allowing one container to serve multiple purposes and reducing the need for separate dedicated containers for each function.
2Reliability
If separate containers are created for each function request, then function execution isolation is maintained, but response time increases due to container initialization overhead
Solution Approach 1:
The container and its execution environment are prepared in advance before function requests arrive. Multiple handler processes are pre-initialized within the container, so when requests come in, they can be immediately assigned to ready handlers without waiting for container creation and initialization, thus reducing response time while maintaining isolation.
Solution Approach 2:
Instead of creating new containers for each request, the system creates copies of handler processes within the existing container. These handler copies inherit the pre-initialized execution environment, avoiding the time-consuming container startup process while maintaining functional isolation through process-level copying.
3Reliability
If each function has dedicated memory resources, then memory isolation is ensured, but memory usage increases due to redundant allocation
Solution Approach 1:
Multiple functions share a common memory space within the single container instead of each having dedicated memory. This eliminates redundant memory allocation for identical data structures and libraries across multiple containers, reducing total memory usage while maintaining isolation through controlled access mechanisms in the shared memory.
4Reliability
If synchronous wait patterns are used for remote data access, then data consistency is maintained, but idle time increases due to blocking calls
Solution Approach 1:
The handler scheduling system dynamically manages the execution state of handlers, allowing them to transition between running, waiting, and suspended states. When a handler needs to perform remote data access, it can be suspended rather than blocking, allowing the dispatcher to allocate CPU resources to other ready handlers, thus reducing idle time while maintaining data consistency through proper synchronization mechanisms.
Data Source
AI summary
A computer implemented method manages function execution in a container. A dispatcher in the container running in a computer system executes a function initialization in response to a first request for a function. The dispatcher in the container running in the computer system creates group of handlers in response to receiving a group of requests for the function. The dispatcher in the container running in the computer system sends the group of requests to the group of handlers in response to receiving the group of requests. The dispatcher in the container running in the computer system executes the group of requests using the group of handlers.