Native-Image In-Memory Cache Bypassing OS Page Cache

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In containerized environments, serialized heaps require unnecessary input/output operations to memory map, leading to inefficient use of resources and compute power, especially when multiple containers do not share the same memory layout, resulting in subpar processing performance and wasted resources.

Innovation Solution

Bypassing the operating system page cache, native-image applications access an in-memory cache shared through inter-process-communication, minimizing input/output and eliminating the need for memory mapping, thus saving system resources and compute power without relying on exact memory layout matches.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If serialized heaps are memory mapped in containerized environments, then data sharing between processes is enabled, but unnecessary input/output operations occur and system resources are wasted

Engineering Contradiction:

Improveprocessing efficiencyVSAvoidsystem resource waste

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The patent extracts the AOT compiled application image from the serialized heap file and loads it directly into memory as an in-memory cache, bypassing the file system and eliminating unnecessary I/O operations associated with memory mapping serialized heaps

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The application image is pre-compiled into a serialized heap file format, then pre-loaded into memory as an in-memory cache before execution, so that the runtime system can directly access the compiled code without performing I/O operations during container startup

Inventive Principle:

Principle #10Preliminary action

2Ease of operation

If operating system page cache is used for AOT applications, then memory management is simplified, but input/output operations increase and processing performance decreases

Engineering Contradiction:

Improvememory management simplicityVSAvoidprocessing performance

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The patent introduces an in-memory cache as an intermediary layer between the AOT compiled application image and the operating system page cache. This cache holds the application image in memory, allowing direct access without requiring I/O operations through the OS page cache, thus improving processing performance while maintaining simple memory management

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If memory mapping is performed for serialized heaps, then data can be shared across processes, but compute power is wasted on unnecessary operations

Engineering Contradiction:

Improvedata sharing capabilityVSAvoidcompute power consumption

Core Design Contradiction:

Adaptability or versatilityVSPower

Solution Approach 1:

The patent merges the application image loading process with the in-memory cache population process. The AOT compiled application image is loaded directly into the in-memory cache from the serialized heap file, combining data loading and caching operations into a single efficient process that enables inter-process communication without separate memory mapping operations

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS11954510B2Native-image in-memory cache for containerized ahead-of-time applications

Publication Date: 2024.04.09 RED HAT INC

AI summary

A method includes identifying an ahead-of-time (AOT) native-image application to be compiled and during AOT compilation of the AOT native-image application; bypassing an operating system page cache corresponding to the AOT native-image application; and accessing, by a processing device, the native-image application from an in-memory cache shared using inter-process-communication.