Serverless Container Traffic Forwarding for Zero-Delay Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing serverless architecture-based cloud platforms rely on centralized gateways for expansion, leading to increased resource overhead, performance loss, and high error rates due to complex configuration and unpredictable burst traffic, which cannot be timely addressed.
Innovation Solution
Detect indicators in the container instance processing flow to determine if they exceed thresholds, and if so, forward service traffic to pre-warmed new container instances to achieve almost zero-delay expansion, avoiding reliance on centralized gateways and reducing error rates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a centralized gateway is used for expansion in serverless architecture, then traffic can be routed to new container instances, but resource overhead and performance loss increase due to gateway operation and maintenance
Solution Approach 1:
The patent extracts the expansion functionality from the centralized gateway and implements it directly in the container instances through peer-to-peer communication. Each container instance can independently receive traffic and route to other instances, eliminating the gateway's expansion management burden while maintaining expansion capability.
Solution Approach 2:
Container instances perform their own expansion operations by maintaining a service registry and automatically routing traffic to new instances. The system serves itself without requiring centralized gateway intervention for instance creation, deletion, or traffic redirection, reducing gateway resource overhead.
2Adaptability or versatility
If a centralized gateway is used for expansion, then traffic routing is possible, but persistent performance loss occurs
Solution Approach 1:
The expansion management functions are extracted from the centralized gateway and distributed to individual container instances. This eliminates the gateway's performance bottleneck by allowing direct peer-to-peer traffic routing between instances, improving overall system performance while maintaining expansion capability.
Solution Approach 2:
The centralized gateway's monolithic expansion management is segmented into distributed functions across multiple container instances. Each instance independently handles its own traffic routing and expansion logic, eliminating the single point of failure and performance limitation inherent in centralized gateway architecture.
3Reliability
If expansion policies are finely configured, then expansion control is precise, but configuration becomes complicated
Solution Approach 1:
Container instances automatically discover and register themselves with the service registry, eliminating the need for manual configuration of expansion policies. The system self-configures by having instances publish their service endpoints and capabilities, allowing precise control without complex user configuration.
Solution Approach 2:
The patent changes the configuration approach from static policy definitions to dynamic parameter registration. Instances automatically provide parameters such as service endpoints, capacity, and routing rules to the registry, enabling precise expansion control through runtime parameter changes rather than pre-configured policies.
4Quantity of substance
If new container instances are created after expansion is triggered, then resource utilization is optimized, but cold start delay affects responsiveness
Solution Approach 1:
The patent implements preliminary action by pre-creating container instances in a suspended or warm state before they are needed. These instances are prepared in advance with their runtime environment loaded but not actively processing traffic, so when expansion is triggered, they can activate immediately without cold start delay while maintaining optimized resource utilization.
Solution Approach 2:
The system performs preliminary anti-action by preemptively creating and warming up container instances before traffic bursts occur. This anticipatory approach counteracts the potential cold start delay by ensuring instances are ready to accept traffic immediately when needed, while resources are only fully allocated when the instances become active.
Data Source
AI summary
Embodiments of the present disclosure provide a method for flexible expansion of a serverless architecture-based cloud service, an apparatus, and a storage medium. A first indicator in a process of sending service traffic from an upstream container instance to a target downstream container instance and/or a second indicator in a process of processing the service traffic by the target downstream container instance are detected in a process of a container instance processing flow in a serverless architecture-based cloud platform, and whether the first indicator and/or the second indicator exceed a corresponding preset threshold is determined; and if so, a part of the service traffic is forwarded to a pre-created and warmed-up new container instance corresponding to the target downstream container instance.