User-Transparent Block Cache for Out-of-Core Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing out-of-core systems face efficiency and scalability issues due to cache misses, particularly with high-performance nonvolatile memory express (NVMe) solid state drives, and implementing user-space block caches with virtual memory interfaces is complex, requiring manual code modifications and optimizations.
Innovation Solution
A user-transparent block cache system using a multi-level Software Address Translation Cache (SATC) with a shared and private cache hierarchy, managed by a runtime module that performs automatic address translations and concurrency control, allowing seamless operation without manual code rewriting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If OS page cache is used for transparent caching, then user transparency is provided, but performance bottlenecks occur due to cache misses with high-performance NVMe SSD arrays
Solution Approach 1:
The patent introduces a user-space block cache as an intermediary layer between the OS page cache and NVMe SSD arrays. This mediator handles caching operations in user space, avoiding the performance bottlenecks of kernel-space page caches while maintaining user transparency through automatic address translation via SATC.
Solution Approach 2:
The patent segments the caching functionality into distinct components: OS page cache for basic transparency, user-space block cache for performance optimization, and SATC for address translation. This segmentation allows each component to operate optimally in its own space without interfering with others.
2Productivity
If custom block caching components are implemented in user space, then performance issues are addressed, but code modifications and implementation complexity are required
Solution Approach 1:
The SATC system provides self-service by automatically performing address translations and managing cache operations without requiring application developers to write custom caching code. The system serves itself by handling pin/unpin operations and address translation transparently.
Solution Approach 2:
The user-space block cache with SATC provides universal caching functionality that works with any application requiring out-of-core processing. It combines address translation, caching, and concurrency control in a single multi-functional system that eliminates the need for application-specific cache implementations.
3Reliability
If pin and unpin operations are performed for every memory access, then data residency is ensured, but efficiency is reduced due to storage-to-memory address translation and concurrency control overhead
Solution Approach 1:
The SATC performs address translations and concurrency control operations in advance, before actual data access occurs. By pre-translating addresses and managing cache state, the system ensures data residency is maintained without requiring pin/unpin operations on every memory access.
Solution Approach 2:
The SATC acts as an intermediary between applications and the block cache, handling all pin/unpin operations and address translations. This mediator absorbs the overhead of concurrency control and address translation, preventing these operations from becoming bottlenecks during actual data access.
4Ease of operation
If a virtual memory interface is provided for user-space block cache, then ease of use is improved, but address translation complexity increases
Solution Approach 1:
The SATC serves as an intermediary that handles the complexity of address translation while presenting a simple virtual memory interface to applications. It translates virtual addresses to physical addresses and manages cache block mappings automatically, hiding the translation complexity from users.
Solution Approach 2:
The SATC performs address translation automatically without user intervention. The system serves itself by maintaining translation tables and performing address conversions transparently, allowing applications to use simple virtual memory interfaces while the SATC handles the complex translation operations in the background.
Data Source
AI summary
Described herein are methods and systems for out-of-core processing of in-memory programs through user-transparent block cache comprising initiating a user program. Application code with memory instructions is initiated. A low level virtual machine or LLVM compiler reads and loads the memory and inserts a software translation call before the memory instructions. The software translation call with memory instructions are received by runtime module with multi-level cache that is built on top of shared cache, and load data blocks using the caches.