Pre-caching Data for Virtual Machine Startup Delays

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Developers face challenges in efficiently utilizing virtual machine resources, including delays in instance startup, over- or under-provisioning, and fair resource allocation, especially when handling high-frequency requests in cloud computing environments.

Innovation Solution

A system maintains a pool of pre-initialized virtual machine instances with loaded operating systems and language runtimes, allowing for immediate use upon request, and employs a message queue service that pre-fetches data needed for event messages to minimize execution time and optimize resource utilization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If virtual machine instances are initialized on-demand when requests arrive, then resource allocation flexibility is improved, but startup delay increases

Engineering Contradiction:

Improveresource allocation flexibilityVSAvoidstartup delay

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The system pre-initializes virtual machine instances in a warming pool before requests arrive, loading operating systems and language runtimes in advance. This preliminary action eliminates startup delays when requests are received, as instances are already prepared and ready for immediate execution.

Inventive Principle:

Principle #10Preliminary action

2Productivity

If more virtual machine instances are maintained in the warming pool, then execution time is reduced, but resource consumption increases

Engineering Contradiction:

Improveexecution timeVSAvoidresource consumption

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The system dynamically adjusts the size of the warming pool based on request patterns and resource availability. Instances are added to or removed from the warming pool as needed, allowing the system to optimize between execution speed and resource consumption rather than maintaining a fixed pool size.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes operational parameters such as the number of instances in the warming pool, cache sizes, and pre-fetching thresholds based on system conditions. This allows optimization of execution time while controlling resource consumption by adapting parameters to current workload demands.

Inventive Principle:

Principle #35Parameter changes

3Speed

If data is pre-fetched for event messages, then processing speed is improved, but system complexity increases

Engineering Contradiction:

Improveprocessing speedVSAvoidsystem complexity

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

The system pre-fetches data needed for event messages before the actual processing occurs. This preliminary data retrieval eliminates wait times during message processing, improving overall processing speed while the complexity is managed through automated cache management.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11228658B1Pre-caching data for use upon execution of program code

Publication Date: 2022.01.18 AMAZON TECH INC

AI summary

Systems and methods for processing requests to execute a program code of a user use a message queue service to store requests when there are not enough resources to process the requests. The message queue service determines whether a request to be queued is associated with data that the program code needs in order to process the request. If so, the message queue service locates and retrieves the data and stores the data in a cache storage that provides faster access by the program code to the pre-fetched data. This provides faster execution of asynchronous instances of the program code.