Serverless Function Scaling via Predictive Heuristics

Solution Overview

Problem

Conventional serverless function systems face inefficiencies in resource utilization due to simplistic request-driven scaling methods, leading to extended lifecycles of unnecessary serverless functions and increased cold start latency when new functions are invoked.

Innovation Solution

The system intelligently scales serverless functions by processing logic that analyzes different types of invocations and metrics such as latencies and idle times to determine optimized scaling rates, using generated scaling heuristics to invoke or remove functions dynamically based on demand.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of time

If request-driven scaling methods are used to manage serverless functions, then serverless function deployment latency is reduced, but computing overhead is wasted on unnecessary functions

Engineering Contradiction:

Improvedeployment latencyVSAvoidcomputing overhead

Core Design Contradiction:

Loss of timeVSLoss of energy

Solution Approach 1:

The patent implements dynamic scaling by adjusting the number of serverless functions based on real-time request patterns and predicted future demand. The system transitions from static or simple request-driven scaling to adaptive scaling that considers temporal patterns, seasonality, and predictive analytics to optimize function deployment and termination decisions.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system performs preliminary actions by predicting future request patterns and proactively scaling functions before peak demand occurs. The patent uses predictive analytics to anticipate future needs and pre-deploy functions, avoiding cold start latency while preventing over-provisioning during low-demand periods.

Inventive Principle:

Principle #10Preliminary action

2Reliability

If serverless function lifecycles are extended to handle requests, then request processing is improved, but unnecessary functions consume resources for extended periods

Engineering Contradiction:

Improverequest processingVSAvoidresource consumption

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The patent implements feedback mechanisms that continuously monitor request patterns, function performance metrics, and resource utilization. This feedback loop enables the system to adjust function lifecycle decisions dynamically, extending lifecycles when needed based on actual demand patterns and terminating functions when predicted demand drops below thresholds.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system changes key parameters including function lifecycle duration, scaling thresholds, and deployment timing based on analyzed request patterns. The patent adjusts these parameters adaptively to match actual usage behavior, transitioning from fixed lifecycle policies to dynamic parameter adjustment that optimizes both reliability and resource efficiency.

Inventive Principle:

Principle #35Parameter changes

3Productivity

If new serverless functions are deployed to handle increased demand, then request capacity is improved, but cold start latency increases for newly invoked functions

Engineering Contradiction:

Improverequest capacityVSAvoidcold start latency

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by predicting future request patterns and proactively scaling functions before peak demand occurs. The patent uses predictive analytics to anticipate future needs and pre-deploy functions, avoiding cold start latency while preventing over-provisioning during low-demand periods.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent implements dynamic scaling by adjusting the number of serverless functions based on real-time request patterns and predicted future demand. The system transitions from static or simple request-driven scaling to adaptive scaling that considers temporal patterns, seasonality, and predictive analytics to optimize function deployment and termination decisions.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS11831410B2Intelligent serverless function scaling

Publication Date: 2023.11.28 RED HAT LLC

AI summary

A system and method of serverless function scaling is described. The method may include receiving one or more types of serverless function invocations. The method also includes invoking one or more serverless functions and receiving metrics from the one or more serverless functions. The method also includes generating scaling heuristics for scaling of a number of serverless functions in view of the received metrics.