Database Replication via In-Memory Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database replication methods face performance bottlenecks and latency issues due to the size of data records and input/output operations, which hinder effective data protection and disaster recovery solutions.
Innovation Solution
A system and method for replicating I/O logs from a primary node to a replication node, utilizing compression and in-memory operations to improve performance, where data pages and metadata pages are processed and transmitted efficiently, enabling active and synchronizing replication modes to ensure data integrity and availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If physical replication of database is implemented, then data protection and disaster recovery are improved, but performance bottleneck occurs due to size of data records and I/O operations
Solution Approach 1:
The patent segments the database replication process into two distinct modes: active replication for real-time data protection and synchronizing replication for periodic consistency verification. This segmentation allows the system to optimize performance by using in-memory operations for active replication while using compression for synchronizing replication, thereby resolving the contradiction between reliability and productivity.
Solution Approach 2:
The patent changes the parameter of data representation by implementing compression algorithms for data pages during synchronizing replication. This parameter change reduces the size of transmitted data, thereby improving replication performance while maintaining data protection capabilities.
2Reliability
If data records are transmitted in original format, then data integrity is maintained, but latency from I/O operations increases
Solution Approach 1:
The patent applies compression to data pages during transmission in synchronizing replication mode, changing the parameter of data size while maintaining data integrity through proper compression and decompression processes. This reduces I/O latency by decreasing the amount of data that needs to be read from and written to storage devices.
Solution Approach 2:
The patent creates compressed copies of data pages for transmission during synchronizing replication, rather than transmitting original uncompressed data. This copying approach with compression reduces the time required for data transmission while maintaining the ability to restore original data integrity at the destination.
3Productivity
If compression is applied to data pages, then data size is reduced and transmission efficiency is improved, but processing complexity increases
Solution Approach 1:
The patent implements dynamic selection of replication modes (active vs. synchronizing) based on system conditions and requirements. The compression processing is dynamically applied only during synchronizing replication mode, allowing the system to balance transmission efficiency needs against processing complexity by enabling compression only when beneficial.
4Speed
If in-memory operations are used, then replication speed is improved, but memory resource consumption increases
Solution Approach 1:
The patent dynamically switches between active replication mode (using in-memory operations for speed) and synchronizing replication mode (using compression for efficiency). This dynamic approach allows the system to leverage in-memory processing benefits when needed while managing memory resource consumption by transitioning to disk-based compression operations when appropriate.
Data Source
AI summary
Disclosed herein are system, method, and computer program product embodiments for replication of database. In one embodiment, one or more data pages generated from a transaction are obtained at a first node. The one or more data pages are compressed. The compressed data pages are added into a first queue in a memory of the first node. The first queue includes a plurality of blocks. A first block of the compressed data pages in the first queue is transmitted to a second node when the first block of the compressed data pages becomes available for replication. The first block of the compressed data pages is stored in a persistent storage of the first node.