In-Memory Database Object Storage Segmentation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In-memory database systems face performance issues when storing large objects (LOBs) due to high memory consumption, as they require significant main memory space, leading to slowed performance, especially for data sizes ranging from megabytes to gigabytes.

Innovation Solution

Implement a method to determine the size of objects and store smaller ones in main memory while moving larger objects to disk storage, using a global container identifier for disk storage, allowing efficient referencing and reducing main memory usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If large objects are stored in main memory of an in-memory database system, then data access speed is improved, but main memory consumption increases significantly

Engineering Contradiction:

Improvedata access speedVSAvoidmain memory consumption

Core Design Contradiction:

SpeedVSQuantity of substance

Solution Approach 1:

The patent segments data storage by size threshold, dividing objects into small objects (stored in main memory) and large objects (stored on disk). This segmentation resolves the contradiction by allowing fast access for small data while avoiding excessive memory consumption for large data, achieving a balanced storage architecture that optimizes both speed and memory usage.

Inventive Principle:

Principle #1Segmentation

2Productivity

If all objects are stored in main memory, then data retrieval performance is improved, but storage costs increase

Engineering Contradiction:

Improvedata retrieval performanceVSAvoidstorage cost

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The patent applies local quality by assigning different storage locations based on object characteristics (size). Small objects that require frequent access remain in expensive but fast main memory, while large objects are stored in cheaper disk storage. This localized quality differentiation resolves the contradiction between retrieval performance and storage cost by optimizing the storage medium for each data type.

Inventive Principle:

Principle #3Local quality

3Loss of time

If large objects are stored in main memory, then data access time is reduced, but overall system performance slows down

Engineering Contradiction:

Improvedata access timeVSAvoidoverall system performance

Core Design Contradiction:

Loss of timeVSProductivity

Solution Approach 1:

The patent implements a dynamic storage strategy where the system automatically determines the optimal storage location (main memory or disk) based on object size thresholds. This dynamic approach resolves the contradiction by adaptively managing storage resources, ensuring that small objects benefit from fast memory access while large objects are efficiently handled on disk, thereby maintaining overall system performance.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS9483513B2Storing large objects on disk and not in main memory of an in-memory database system

Publication Date: 2016.11.01 SAP SE

AI summary

A method, computer program product and system are provided. The method, computer program product and system execute a process for determining a size of an object, the object having raw data that is operable upon by one or more physical operators. If the object is smaller than a threshold size, the object is stored in main memory of an in-memory database system. If the object is equal to or larger than the threshold size, the object is stored in a persistency of a disk storage, where storing the object in a disk storage further includes generating a global container identifier (ID) for the object, the global container ID referencing raw data of the object stored in the persistency of the disk storage.