Virtual Machine Snapshots for On-Demand Code Execution Latency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The existing on-demand code execution systems face high latency due to the need to boot and initialize virtual machine instances for each task, which consumes computing resources and increases 'cold start' times, making it inefficient to execute tasks in real-time.

Innovation Solution

The system maintains virtual machine instances in an initialized but non-executing state, generates snapshots of these instances at specific points, and uses cascading snapshots to reduce data storage requirements, allowing for rapid initialization and execution of tasks by referencing prior machine states.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If virtual machine instances are booted and initialized for each task execution, then tasks can be executed with full computing resources, but task execution latency increases and computing resources are consumed during initialization

Engineering Contradiction:

Improvetask execution capabilityVSAvoidcold start time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary initialization of virtual machine instances before actual task execution. Virtual machine instances are pre-configured with operating systems, runtime environments, and necessary software configurations, then saved as machine states. When a task arrives, the system restores a pre-initialized virtual machine from its saved state, eliminating the need to boot and initialize the virtual machine at task execution time, thus reducing cold start latency while maintaining full execution capability

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system creates and maintains copies of virtual machine machine states that represent pre-initialized configurations. Instead of booting virtual machines from scratch for each task, the system copies restored virtual machine states and reuses them for subsequent task executions. This copying mechanism allows rapid deployment of initialized virtual machines without repeating the lengthy initialization process, thereby reducing execution latency while preserving reliable task execution

Inventive Principle:

Principle #26Copying

2Productivity

If virtual machine instances are maintained in initialized state, then task execution speed improves, but computing resource consumption increases

Engineering Contradiction:

Improvetask execution speedVSAvoidcomputing resource usage

Core Design Contradiction:

ProductivityVSUse of energy by moving object

Solution Approach 1:

The system creates lightweight, temporary virtual machine instances from saved machine states only when tasks need execution, uses them for the specific task, and then terminates them. These virtual machines are not maintained in a continuously running initialized state but are instead created on-demand from saved states, used briefly for task execution, and then disposed of. This approach enables fast task execution when needed while avoiding continuous resource consumption during idle periods

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

Solution Approach 2:

The system dynamically creates and destroys virtual machine instances based on task arrival and completion. Rather than maintaining a static pool of continuously running virtual machines, the system adapts the number and state of virtual machines to actual workload demands. Virtual machines are restored from saved states when tasks arrive and terminated when tasks complete, optimizing the balance between execution speed and resource consumption according to real-time system conditions

Inventive Principle:

Principle #15Dynamics

3Loss of time

If full virtual machine states are stored for rapid restoration, then initialization time is reduced, but storage requirements increase

Engineering Contradiction:

Improveinitialization timeVSAvoiddata storage requirements

Core Design Contradiction:

Loss of timeVSQuantity of substance

Solution Approach 1:

The system segments the virtual machine state into multiple components: a base machine state containing the operating system and common configurations, and incremental updates containing task-specific configurations and changes. Instead of storing complete duplicate copies of entire virtual machine states for each task, the system stores the base state once and maintains separate incremental update records for each task. During restoration, the base state is loaded first, then incremental updates are applied sequentially, achieving rapid initialization while minimizing storage requirements through this segmented approach

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11836516B2Reducing execution times in an on-demand network code execution system using saved machine states

Publication Date: 2023.12.05 AMAZON TECH INC

AI summary

Systems and methods are described for reducing latency to service requests to execute code on an on-demand code execution system by maintaining snapshots of virtual machine instances in a ready state to execute such code. A user may submit code to the on-demand code execution system, which code depends on other software, such as an operating system or runtime. The on-demand code execution system can generate a virtual machine instance provisioned with the other software, and initialize the instance into a state at which it is ready to execute the code. The on-demand code execution system can then generate a snapshot of the state of the instance, and halt the instance. When a request to execute the code is received, the snapshot can be used to quickly restore the instance. The code can then be executed within the instance, reducing the need to initialize the instance or maintain the instance in an executing state.