Persistent Helper Functions for Serverless Latency Reduction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Serverless computing models like FaaS face limitations such as cold starts, timeout limits, provisioning overhead, and inefficient pricing models, which hinder performance and scalability, especially for long-running operations and computing-intensive tasks.

Innovation Solution

The introduction of Persistent Helper Functions (PHFs) in a Result-as-a-Service (RaaS) model, which allows stateful, continuously trainable, and scalable functions that can be executed on GPUs or ASICs, with customizable billing and subscription models, enabling precomputation and result caching to reduce latency and computational overhead.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If serverless computing uses dynamic resource allocation with container creation for each function invocation, then flexibility and scalability are improved, but cold start latency and provisioning overhead increase

Engineering Contradiction:

ImproveflexibilityVSAvoidcold start latency

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by pre-warming containers before they are actually needed. The container management system proactively creates and initializes containers during low-utilization periods, so that when function invocations occur, the containers are already ready to execute, eliminating cold start latency while maintaining the flexibility of dynamic allocation

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically adjusts container lifecycle management based on real-time utilization patterns and function invocation rates. The container management system monitors usage patterns and adapts container creation, warming, and termination timing to balance between reducing cold starts and maintaining flexibility for varying workloads

Inventive Principle:

Principle #15Dynamics

2Productivity

If cloud providers set timeout limits for function execution, then resource management and cost control are improved, but long-running batch operations cannot be executed

Engineering Contradiction:

Improveresource management efficiencyVSAvoidexecution duration

Core Design Contradiction:

ProductivityVSDuration of action of moving object

Solution Approach 1:

The system segments long-running batch operations into multiple smaller function invocations that can execute within timeout limits. The batch operation is divided into discrete tasks that can be processed in chunks, with each chunk fitting within the provider's timeout window, while the overall operation completes successfully through coordinated multiple invocations

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system maintains continuity of useful action by implementing persistent state management across function invocations. When batch operations are split into multiple invocations, the state and progress are preserved between invocations, allowing the operation to continue from where it left off without losing computational progress, thus enabling long-running operations to complete despite timeout constraints

Inventive Principle:

Principle #20Continuity of useful action

3Speed

If containers are reused for subsequent function invocations within a time period, then execution speed is improved, but state information must be separately maintained in state databases

Engineering Contradiction:

Improveexecution speedVSAvoidstate management complexity

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

The system merges state management back into the container by implementing persistent storage mechanisms at the container level. This allows containers to retain state information across invocations without requiring separate state database management, combining the benefits of container reuse for speed with simplified state management, reducing the overall system complexity

Inventive Principle:

Principle #5Merging (Combining)

4Loss of energy

If a library of persistent helper functions is introduced to optimize FaaS offerings, then computational overhead and costs are reduced, but system complexity increases

Engineering Contradiction:

Improvecomputational costVSAvoidsystem complexity

Core Design Contradiction:

Loss of energyVSDevice complexity

Solution Approach 1:

The system introduces persistent helper functions as intermediary components that sit between the event sources and the main FaaS functions. These helper functions perform preparatory computations, data validation, and result caching, reducing the computational burden on the main functions and lowering overall computational costs while managing complexity through a clear layered architecture

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20240385899A1Function Delivery Network for Serverless Applications

Publication Date: 2024.11.21 MADISETTI VIJAY

AI summary

A method for improving performance of functions-as-a-service including receiving at a function delivery network a first function call from a first user and a second function call from a second user calling the first function within a first time window. The method further includes sending a first function call request to a cloud service responsive to the first and second function calls, performing at the cloud service a first function responsive to the first function call request producing a first function result, receiving at the function delivery network the first function result, and transmitting from the function delivery network the first function result to each of the first user and the second user.