Persistent Helper Functions for Serverless Cold Start Latency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Serverless computing models like FaaS face limitations such as cold starts, timeout limits, provisioning overhead, and inefficient pricing models, which hinder performance and scalability in cloud environments.

Innovation Solution

The introduction of Persistent Helper Functions (PHFs) in a Result-as-a-Service (RaaS) model, which allows for stateful operations, elastic scaling, continuous training, and customizable billing, enabling precomputation and data sharing across functions to reduce latency and computational costs.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Use of energy by moving object

If FaaS executes functions by dynamically allocating resources, then resource utilization is improved, but cold start latency increases

Engineering Contradiction:

Improveresource utilizationVSAvoidcold start latency

Core Design Contradiction:

Use of energy by moving objectVSLoss of time

Solution Approach 1:

The system performs preliminary actions by pre-computing results for anticipated function calls and storing them in a result cache before they are actually needed. When a function call arrives, the system first checks the cache for pre-computed results, avoiding the need to execute the function from scratch and thus eliminating cold start latency for cached operations.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system creates copies of computation results and stores them in a persistent result cache. Instead of re-executing functions, the system retrieves and returns copied results from the cache, significantly reducing execution time and eliminating the need to re-initialize containers for repeated function calls with identical parameters.

Inventive Principle:

Principle #26Copying

2Productivity

If FaaS sets timeout limits for function execution, then resource management is improved, but long-running batch operations cannot complete

Engineering Contradiction:

Improveresource management efficiencyVSAvoidfunction execution duration

Core Design Contradiction:

ProductivityVSDuration of action of moving object

Solution Approach 1:

The system segments long-running batch operations into smaller sub-tasks that can be executed within timeout limits. Each sub-task produces intermediate results that are stored in the persistent cache, allowing the overall computation to progress across multiple timeout cycles without losing state or requiring re-initialization.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The persistent result cache ensures continuity of useful action by preserving computation results across timeout boundaries. When a long-running operation is interrupted by a timeout, the system can resume execution from where it left off, retrieving previously computed results from the cache rather than starting over, thus maintaining continuous progress on batch operations.

Inventive Principle:

Principle #20Continuity of useful action

3Reliability

If FaaS creates a new container for each function invocation, then function isolation is improved, but execution overhead increases

Engineering Contradiction:

Improvefunction isolationVSAvoidcontainer provisioning overhead

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system merges multiple function executions into a single persistent container by implementing a result cache that survives container lifecycle events. Instead of creating new containers for each invocation, the system maintains computation state and results within the same container instance, reducing provisioning overhead while preserving isolation through cached result separation.

Inventive Principle:

Principle #5Merging (Combining)

4Loss of time

If FaaS reuses containers for subsequent invocations, then cold start latency is reduced, but state persistence becomes problematic

Engineering Contradiction:

Improvecold start latencyVSAvoidstate persistence

Core Design Contradiction:

Loss of timeVSLoss of information

Solution Approach 1:

The system extracts the persistent state (result cache) from the transient container lifecycle by implementing a separate caching layer that stores computation results independently of container creation and destruction. This allows containers to be reused for rapid execution while the extracted state persists across container boundaries, solving both cold start latency and state persistence issues.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS10983768B2Method and system for persistent helpers for functions as a service (FAAS) in cloud computing environments

Publication Date: 2021.04.20 MADISETTI VIJAY

AI summary

A method for improving the performance of functions-as-a-service including receiving a first function call including a first argument, performing a first function responsive to the first argument including by the first function call, producing a first function result, generating a first precomputation argument that differs from the first argument responsive to the first function call, executing a first persistent function responsive to the first precomputation argument, the persistent function including performing at least one precomputation operation, the at least one precomputation operation including performing the first function responsive to the first precomputation argument, producing a first precomputation result, storing the first precomputation result, receiving a second function call including a second argument, comparing the second argument to the first precomputation argument, and upon determining the second argument is identical to the first precomputation argument, transmitting the first precomputation result from a source of the second function call.