Unified Transactional Data Stream from Disk and In-Memory Logs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database management systems are not adapted to perform log replay over multiple database logs and replay transactions in the originally committed order, especially for database objects residing in both disk-based storage and in-memory row storage, which hinders efficient replication and disaster recovery.
Innovation Solution
A replication system with a primary server and a secondary server that uses system log and IMRS log replicant agents to generate a single transactional data stream from multiple database logs, ensuring that log entries are replayed in the correct order by assigning unique identifiers and coordinating the generation and transmission of stream packages based on shared state information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If separate page logs are maintained for disk-based storage and in-memory row storage, then database operations can be tracked independently, but log replay cannot be performed across multiple logs and transactions cannot be replayed in the originally committed order
Solution Approach 1:
The patent merges multiple separate database logs (system log and IMRS log) into a single unified data stream that can be replayed together. The replication agent combines log entries from both logs, assigns unified sequence numbers, and transmits them as one coordinated stream to the standby server, enabling comprehensive log replay across previously separate log sources.
Solution Approach 2:
The replication agent acts as an intermediary that receives log entries from multiple sources (system log and IMRS log), processes them through a unified sequencing mechanism, and outputs a single ordered data stream. This intermediary component coordinates the log entries and ensures they are transmitted in the correct committed order to the standby server.
2Ease of manufacture
If multiple separate logs are used for different storage types, then specialized handling is possible, but coordination and ordering of transactions across logs becomes complex
Solution Approach 1:
The replication agent is designed with multi-functionality to handle both system log and IMRS log entries through a single unified process. It can identify, sequence, and transmit entries from either log type using the same mechanism, eliminating the need for separate specialized handling paths and reducing overall system complexity.
3Ease of operation
If log entries from multiple logs are transmitted separately, then transmission is straightforward, but the standby server cannot replay transactions in the originally committed order
Solution Approach 1:
The replication agent performs preliminary sequencing and ordering of log entries before transmission to the standby server. It assigns sequence numbers and establishes the correct committed order in advance, so that when the data stream is transmitted, the standby server can replay transactions in the correct order without requiring complex post-reception coordination.
Data Source
AI summary
Disclosed herein are system, method, and computer program product embodiments for generating a single transaction data stream from multiple database logs. An embodiment operates by assigning a first unique identifier to a first log entry associated with a disk store, a second unique identifier to a second log entry associated with in an in memory row store, and a third unique identifier to a third log entry based on the second unique identifier. The embodiment further operates by determining a first stream package based on the first log entry, a second stream package based on the third log entry, and a replay order for the first stream package and the second stream package based on the unique identifiers. In some embodiments, the operation further includes sending the first stream package and the second stream package to a replicant server in accordance with the replay order.