Common Runtime Framework for Database Log Replay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database systems face challenges in maintaining data consistency and availability across primary and secondary systems during load balancing and high availability/disaster recovery operations, where divergent data states can lead to inconsistent responses and increased downtime.
Innovation Solution
A common runtime and recovery framework is implemented to manipulate data by receiving log transaction information from the primary database system, parsing it to identify modified data objects, and replaying these transactions on the secondary system, ensuring data objects are current in both main memory and persistence data volumes, thereby maintaining data consistency and enabling simultaneous utilization of both systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a secondary database system is maintained as a complete copy of the primary system with identical hardware and software requirements, then data consistency and system availability are improved, but hardware costs and system complexity increase
Solution Approach 1:
The patent implements a secondary database system that maintains a complete copy of the primary database in main memory, enabling fast data retrieval and system failover. The copying mechanism ensures data consistency while allowing the secondary system to operate with reduced hardware requirements compared to traditional identical replicas.
Solution Approach 2:
The secondary database system is designed to serve multiple functions: it acts as a standby system for failover, provides read-only query capabilities during normal operation, and maintains data consistency through log replay. This multi-functionality reduces the need for separate dedicated backup systems.
2Reliability
If log transaction information is continuously replayed on the secondary database system to maintain data consistency, then data synchronization is improved, but processing time and system overhead increase
Solution Approach 1:
The secondary database system pre-loads data objects from persistence storage into main memory before they are needed for query processing. This preliminary action reduces the time required to access and process data during normal operation and failover scenarios.
Solution Approach 2:
The log replay process operates continuously in the background on the secondary system, ensuring that data synchronization is maintained without interrupting query processing. This continuous background operation minimizes the impact on system performance while maintaining up-to-date data.
3Speed
If data objects are copied from persistence storage to main memory on the secondary system, then query response time is improved, but memory usage and hardware resources increase
Solution Approach 1:
The secondary database system maintains frequently accessed data objects in main memory while keeping less frequently accessed data in persistence storage. This local quality differentiation optimizes query response time for hot data while managing memory resources efficiently.
Solution Approach 2:
Instead of loading all data objects into main memory, the system selectively loads only those data objects that are currently needed or frequently accessed. This partial loading approach balances query performance with memory resource constraints.
Data Source
AI summary
A computer implemented method for manipulating data comprises receiving log transaction information from a primary database system at a secondary database system and parsing the log transaction information to identify data objects that have been modified on the primary database system. The secondary database system determines whether the data objects are present in main memory on the secondary database system. If the data objects are not in the main memory, they are copied from persistence data volumes to the main memory. The data objects are modified by replaying the log transactions on the data objects in the main memory. Then, the data objects in the persistence data volumes are modified. Related apparatus, systems, techniques and articles are also described.