Recoverable Commit Identifier for Distributed Database Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database recovery protocols fail to efficiently manage commit identifiers for in-doubt transactions in distributed database systems, leading to inaccurate results or data corruption during recovery, especially when transactions are not fully committed prior to interruption.
Innovation Solution
Implementing an in-memory reference at the master node to correlate transaction identifiers with commit identifiers, allowing the master node to assign and store these identifiers, and ensuring their atomic persistence, enabling rapid access and accurate recovery even if the database system is interrupted before the commit log is fully written.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If distributed database system recovers using traditional sequential replay methods, then system recovery can proceed without commit identifiers, but in-doubt transactions cannot be properly identified and data corruption may occur
Solution Approach 1:
The system pre-assigns commit identifiers to transactions at the master node before the slave node completes the commit operation. This preliminary assignment ensures that even if the slave node fails before writing the commit log, the commit identifier exists and can be used during recovery to determine the transaction's committed state, thereby improving recovery accuracy without requiring complex post-failure analysis
Solution Approach 2:
An in-memory reference structure is introduced as an intermediary between the master node's commit log and the slave node's transaction state. This reference stores the mapping between transaction identifiers and commit identifiers, allowing the slave node to quickly determine whether in-doubt transactions were committed without complex log analysis, thus improving reliability while managing complexity through a dedicated intermediary component
2Speed
If the master node maintains an in-memory reference for all transaction commits, then commit identifier access is rapid, but memory usage increases
Solution Approach 1:
The in-memory reference structure stores only the critical mapping information (transaction identifier to commit identifier) needed for rapid recovery determination, rather than maintaining complete transaction state information. This selective storage provides fast access to commit identifiers while limiting memory consumption to only the essential data required for the specific function of commit status verification
Solution Approach 2:
The system extracts only the necessary commit identifier mapping from the complete transaction log data and stores it in-memory. By separating this critical subset of information from the full transaction history, the system achieves rapid access speed for commit identifier lookup while keeping memory usage proportional to the number of active transactions rather than the total transaction history
3Reliability
If the slave node waits for commit log to be fully written before considering transaction committed, then data consistency is maintained, but recovery time increases when interruption occurs
Solution Approach 1:
The commit identifier is assigned to the transaction at the master node in advance, before the slave node completes writing the commit log. This preliminary assignment allows the slave node to have a head start on the commit process without compromising consistency, because the authoritative commit identifier already exists and can be used to verify commit status during recovery, thereby reducing recovery time while maintaining data consistency
Solution Approach 2:
The slave node continuously monitors the commit log for its assigned commit identifier as feedback that the transaction has been committed. This feedback mechanism allows the slave node to determine commit status efficiently without waiting for the entire log to be written, reducing recovery time while maintaining consistency through verification of the authoritative commit identifier from the master node
Data Source
AI summary
Technologies are described for facilitating database system recovery in a distributed database environment having a master node and at least one slave node. The master node maintains an in-memory reference that includes transaction identifiers and their corresponding commit identifiers. When a slave node requests that a commit identifier be assigned to a transaction, the transaction and commit identifiers for the transaction are added to the in-memory reference. The commit identifier is sent by the master node to the slave node. The slave node writes a log entry to a commit log that includes the transaction identifier and the assigned commit identifier. If the database system is interrupted before the slave node writes the commit log entry, the slave node can request the commit identifier for the transaction from the master node, which retrieves the identifier from the in-memory reference.