Serverless Container Expansion Using Pre-Warmed Traffic Forwarding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing serverless architecture-based cloud platforms rely on centralized gateways for expansion, leading to resource overhead, performance loss, and high error rates due to complex configuration and unpredictable burst traffic, which cannot be timely addressed.
Innovation Solution
Detect indicators in the container instance processing flow to determine if they exceed thresholds, and if so, forward service traffic to pre-warmed new container instances, avoiding delays and overloads by creating instances in advance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a centralized gateway is used for expansion in serverless architecture, then expansion capability is achieved, but resource overhead and performance loss increase
Solution Approach 1:
The patent extracts the expansion function from the centralized gateway and implements it directly in the container instances. Each container instance independently detects traffic indicators and decides whether to expand, eliminating the need for centralized gateway coordination and reducing its resource overhead.
Solution Approach 2:
Container instances perform self-service expansion by autonomously detecting traffic indicators, creating new container instances, and forwarding traffic without relying on centralized gateway control. This decentralized self-service approach reduces the gateway's operational burden and resource consumption.
2Adaptability or versatility
If centralized gateway is used for expansion, then expansion function is provided, but operation and maintenance complexity increases
Solution Approach 1:
The expansion function is segmented from the centralized gateway and distributed to individual container instances. Each instance independently handles its own expansion decisions and operations, dividing the complex centralized management task into simpler decentralized units that are easier to operate and maintain.
3Adaptability or versatility
If fine configuration of expansion policies is required, then expansion control is achieved, but configuration complexity increases
Solution Approach 1:
Container instances automatically detect traffic indicators and make expansion decisions based on predefined thresholds, eliminating the need for users to manually configure complex expansion policies. The system self-adjusts based on real-time traffic conditions, simplifying operation while maintaining control.
4Productivity
If expansion is performed without pre-warmed containers, then resource efficiency is improved, but response time to burst traffic increases
Solution Approach 1:
When traffic indicators approach threshold values, the system pre-creates and warms up container instances before actual burst traffic arrives. This preliminary action ensures that when burst traffic occurs, pre-warmed containers are immediately available to handle the load, reducing response time while maintaining resource efficiency through conditional pre-creation.
5Adaptability or versatility
If new container instances are created during burst traffic, then expansion is achieved, but cold start delay occurs
Solution Approach 1:
The system creates and warms up container instances in advance when traffic indicators suggest approaching capacity limits, before the actual burst traffic arrives. This preliminary creation and warming eliminates cold start delays during critical traffic handling, as containers are already ready to serve requests immediately.
6Adaptability or versatility
If centralized gateway manages expansion, then coordination is achieved, but error rate increases due to unpredictable traffic
Solution Approach 1:
The patent segments the expansion decision-making process from the centralized gateway to individual container instances. Each instance independently monitors its own traffic indicators and makes expansion decisions, eliminating the coordination bottlenecks and errors that occur in centralized gateway management during unpredictable burst traffic.
Solution Approach 2:
Container instances continuously detect traffic indicators and provide real-time feedback to make dynamic expansion decisions. This distributed feedback mechanism allows each instance to respond immediately to local traffic conditions, improving reliability by eliminating the coordination delays and errors inherent in centralized gateway management.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Embodiments of the present disclosure provide a method for flexible expansion of a serverless architecture-based cloud service, an apparatus, and a storage medium. A first indicator in a process of sending service traffic from an upstream container instance to a target downstream container instance and/or a second indicator in a process of processing the service traffic by the target downstream container instance are detected (5201) in a process of a container instance processing flow in a serverless architecture-based cloud platform, and whether the first indicator and/or the second indicator exceed a corresponding preset threshold is determined; and if so, a part of the service traffic is forwarded (S202) to a pre-created and warmed-up new container instance corresponding to the target downstream container instance.