Continuous Page Flusher for In-Memory Database Savepoint Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database recovery processes are inefficient due to the need to manage and flush modified pages to physical disk storage, which can slow down savepoint creation and log replay, especially in high-availability systems where the secondary system must operate as quickly as the primary system.
Innovation Solution
Implementing a page flusher that selectively flushes modified pages to physical disk storage based on a pre-defined time window and page type, allowing for continuous or periodic execution, independent of savepoints, to speed up savepoint creation and minimize savepoints during log replay.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all modified pages are flushed to physical disk storage before savepoint creation, then data consistency is ensured, but savepoint creation time increases significantly
Solution Approach 1:
The system performs preliminary flushing of modified pages to disk storage before savepoint creation. The continuous page flusher proactively identifies and flushes pages that were last modified outside a pre-defined time window, ensuring that when savepoint creation is needed, fewer pages remain in memory that require flushing, thus reducing savepoint creation time while maintaining data consistency
Solution Approach 2:
The continuous page flusher operates continuously or periodically in the background, continuously identifying and flushing modified pages to disk storage. This continuous action ensures that pages are gradually moved from memory to disk over time, preventing accumulation of unflushed pages and reducing the burden on savepoint creation operations without compromising data consistency
2Reliability
If frequent page flushing is performed to ensure data durability, then recovery speed improves, but system performance and productivity deteriorate
Solution Approach 1:
The continuous page flusher operates on a periodic basis with a pre-defined time window, flushing pages that were last modified outside this window. This periodic action ensures that pages are flushed at appropriate intervals rather than continuously, balancing recovery preparedness with system performance by avoiding excessive flushing operations while still ensuring data durability
Solution Approach 2:
The system uses a pre-defined time window parameter to determine which pages need flushing. By adjusting this time window parameter, the system can control the frequency and volume of page flushing operations, optimizing the balance between recovery speed and system performance based on specific operational requirements
3Reliability
If savepoints are created frequently to ensure transaction safety, then transaction rollback capability improves, but log replay time increases
Solution Approach 1:
The continuous page flusher performs preliminary flushing of modified pages to disk before savepoint creation. This preliminary action ensures that when a savepoint is created, fewer modified pages remain in memory, reducing the amount of work required during log replay operations and thereby reducing log replay time while maintaining transaction safety through proper savepoint creation
Data Source
AI summary
Execution of a page flusher is initiated in an in-memory database system in which pages are loaded into memory and which has associated physical disk storage. Thereafter, the page flusher identifies pages that were last modified outside a pre-defined time window. The page flusher then flushes the identified modified pages to the physical disk storage.