FaaS Function Instance Retention for Startup Latency Reduction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing FaaS implementations incur increased startup latency due to the ephemeral nature of function instances, which are immediately purged from primary memory after execution, leading to repeated loading and increased execution times for frequently invoked functions and function chains.

Innovation Solution

Implementing a 'sticky function' approach where host systems retain function instances in primary memory after execution, and a 'read-ahead chaining' approach where function manager identifies and pre-loads next functions in commonly invoked chains, reducing the need for repeated loading and improving execution efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If function instances are immediately purged from primary memory after execution, then sufficient free space is maintained in the primary memory pool, but startup latency increases due to repeated loading of function images

Engineering Contradiction:

Improvefree space in primary memory poolVSAvoidstartup latency

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The system pre-loads function images into primary memory before they are needed for execution. The function manager monitors invocation patterns and proactively loads function images from storage into primary memory, so that when a function is invoked, its image is already available in memory, eliminating the startup latency caused by repeated loading.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system changes the state of function images from ephemeral (immediately purged after execution) to persistent (retained in primary memory). By modifying the lifecycle parameter of function instances, the system allows them to remain in memory beyond their execution, transforming them into reusable cached objects that reduce startup latency for subsequent invocations.

Inventive Principle:

Principle #35Parameter changes

2Loss of time

If function instances are retained in primary memory after execution, then startup latency is reduced for frequently invoked functions, but free space in the primary memory pool is consumed

Engineering Contradiction:

Improvestartup latencyVSAvoidfree space in primary memory pool

Core Design Contradiction:

Loss of timeVSQuantity of substance

Solution Approach 1:

The system applies different retention policies to different function instances based on their invocation patterns. Frequently invoked functions have their images retained in primary memory, while less frequently invoked functions are allowed to be purged. This local differentiation optimizes the balance between memory utilization and startup latency reduction.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The system retains function images in primary memory beyond the minimum required period. Instead of purging images immediately after execution or only after a short timeout, the system keeps them retained for an extended period or until memory pressure occurs, providing a buffer that ensures startup latency is reduced even under varying load conditions.

Inventive Principle:

Principle #16Partial or excessive action

3Productivity

If function images are reloaded into primary memory for each invocation, then memory resources are efficiently managed, but execution time of function chains increases

Engineering Contradiction:

Improvememory resource efficiencyVSAvoidexecution time of function chains

Core Design Contradiction:

ProductivityVSDuration of action of moving object

Solution Approach 1:

The system maintains function images in primary memory continuously across multiple invocations and function chain executions. Instead of allowing memory to be cleared and refilled repeatedly, the system sustains the presence of function images in memory, ensuring that subsequent functions in a chain can execute without interruption or reloading delays, thereby maintaining continuous useful action.

Inventive Principle:

Principle #20Continuity of useful action

Solution Approach 2:

The system pre-loads function images that are part of commonly invoked function chains into primary memory in advance. By analyzing invocation patterns and identifying function chains, the system proactively loads all necessary function images for a chain before execution begins, ensuring that no function in the chain experiences startup latency due to image loading.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20240362030A1Reducing the startup latency of functions in a faas infrastructure

Publication Date: 2024.10.31 VMWARE INC

AI summary

Techniques for reducing the startup latency of functions in a Functions-as-a-Service (FaaS) infrastructure are provided. In one set of embodiments, a function manager of the FaaS infrastructure can receive a request to invoke a function uploaded to the infrastructure and can retrieve information associated with the function. The retrieved information can include an indicator of whether instances of the function may be sticky (i.e., kept in host system primary memory after function execution is complete), and a list of zero or more host systems in the FaaS infrastructure that currently have an unused sticky instance of the function in their respective primary memories. If the indicator indicates that instances of the function may be sticky and if the list identifies at least one host system with an unused sticky instance of the function in its primary memory, the function manager can select the at least one host system for executing the function.