Container Pool Scaling Without Restart
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for deploying and scaling user applications in containers are slow, often requiring minutes and leading to downtime or wasteful resource reservation, especially in fluctuating compute workloads for applications like machine learning and direct-to-consumer services.
Innovation Solution
A method and system for rapid deployment and scaling of user applications by initializing a pool of containers with minimal compute resources, allowing for on-the-fly scaling of selected containers without restarts, using a shared filesystem for efficient resource allocation and reducing downtime.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If traditional container deployment methods are used, then applications can be deployed with full compute resources, but deployment time is long (minutes) and causes downtime
Solution Approach 1:
The system pre-initializes a pool of containers with minimal compute resources before deployment is needed. These containers are created in advance with basic runtime environment but without full application loading, allowing them to be ready for rapid scaling when deployment requests arrive, thus eliminating deployment wait time and downtime.
Solution Approach 2:
The container initialization process is segmented into phases: initial container creation with minimal resources, and subsequent resource allocation when needed. This allows the system to separate the time-consuming resource provisioning from the deployment moment, achieving fast deployment by having containers ready in a semi-initialized state.
2Productivity
If compute resources are reserved in advance for deployment, then deployment can be fast, but resource wastage occurs when resources remain unused
Solution Approach 1:
The system dynamically adjusts compute resource allocation based on actual deployment needs. Containers in the pool start with minimal resources and receive additional resources on-demand when scaling is required. This dynamic allocation eliminates static resource reservation waste while maintaining fast deployment capability, as resources are only consumed when actually needed for running applications.
Solution Approach 2:
The system changes compute resource parameters (CPU, memory, storage) of containers in the pool based on their current state and deployment requirements. Containers can be rapidly scaled from minimal to full resources without permanent allocation, allowing the system to adapt resource levels to actual workload demands and avoid continuous resource wastage.
3Ease of operation
If containers are scaled up by creating new containers, then resource allocation is simple, but scaling time is long and compute resources are exhausted
Solution Approach 1:
The system performs preliminary container initialization work in advance, creating a pool of pre-configured containers with basic runtime environments. When scaling is needed, the system simply activates and allocates resources to these pre-prepared containers rather than creating them from scratch, dramatically reducing scaling time and avoiding compute resource exhaustion during peak deployment periods.
Solution Approach 2:
The system uses template-based container images to rapidly replicate container configurations. Pre-initialized containers serve as templates that can be quickly instantiated and scaled, maintaining operational simplicity while achieving fast scaling through efficient copying of proven container configurations rather than de novo creation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method for deploying or scaling a user application in a container execution system comprising a plurality of nodes. The method comprises initializing a pool of a plurality of containers running on one or more of the nodes, each container of the pool having compute resources specified at a first lower level, and receiving a request for deploying or scaling up the user application. One of the nodes is selected which has sufficient additional compute resources for running a new instance of the user application and one of the containers of the pool running on the selected node is selected. The selected container is scaled up without restarting the container, by increasing the compute resources specified for the selected container to a second higher level to generate a scaled-up container, wherein the scaling is based on one or more scaling parameters, and the user application is run in the newly scaled-up container on the selected node.