In-Memory Row Storage for Database Access Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database management systems face challenges in achieving extreme online transaction processing (OLTP) performance on commodity hardware with low total cost of ownership, particularly in leveraging the high-end power of modern multi-core platforms with large memory capacities.
Innovation Solution
The implementation of an in-memory row storage architecture that dynamically monitors access patterns to database rows, determining the most suitable location for database operations between a persistent page store and an in-memory row store based on workload parameters, and strategically migrating data between the two to optimize performance and memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is stored in persistent page store, then data durability is ensured, but access speed deteriorates
Solution Approach 1:
The patent segments the storage system into two distinct portions: a persistent page store for durability and an in-memory row store for speed. This segmentation allows different data to be stored in different locations based on access patterns, resolving the contradiction between durability and access speed by providing both storage options simultaneously.
Solution Approach 2:
The in-memory row store acts as an intermediary between the persistent page store and the database operations. Frequently accessed rows are moved to this intermediate in-memory layer, which provides fast access while the persistent store maintains durability. This intermediary layer resolves the speed-durability contradiction by caching hot data in memory.
2Speed
If in-memory row store is used, then access speed is improved, but memory resource consumption increases
Solution Approach 1:
The patent applies local quality by storing only frequently accessed rows in the in-memory portion, while less frequently accessed rows remain in the persistent page store. This selective approach ensures that memory resources are consumed only for data that benefits from fast access, optimizing the balance between speed improvement and memory consumption.
Solution Approach 2:
The system dynamically monitors access patterns and automatically moves rows between the in-memory row store and persistent page store based on workload parameters. This dynamic behavior allows the system to adapt memory usage to actual access patterns, ensuring memory is used efficiently for hot data while cold data remains on disk.
3Productivity
If dynamic monitoring and migration is implemented, then OLTP performance is improved, but system complexity increases
Solution Approach 1:
The system implements self-service by automatically monitoring access patterns and performing migrations between storage portions without manual intervention. The database management system autonomously determines workload parameters and selects optimal storage locations, reducing the operational complexity despite the enhanced functionality.
Solution Approach 2:
The system uses feedback from access pattern monitoring to dynamically adjust data placement. By continuously observing workload parameters and using this feedback to make migration decisions, the system optimizes OLTP performance automatically. This feedback mechanism manages complexity by using observed patterns rather than requiring complex predictive models.
Data Source
AI summary
A system for an in-memory row storage architecture can be provided. In some implementations, the system performs operations comprising monitoring access to a plurality of rows in a database partition, a first portion of the plurality of rows stored in a persistent page store and a second portion of the plurality of rows stored in an in-memory row store. The operations can further comprise determining workload parameters based on the monitoring, selecting a location for performing a database operation on the database partition based on the workload parameters, and performing the database operation on the database partition within the selected location (e.g., the persistent page store or the in-memory row store). Related systems, methods, and articles of manufacture are also described.