In-Memory Database Recovery via Reverse Log Scanning

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional in-memory database recovery methods are inefficient and slow due to replaying all log records from the transaction log, leading to prolonged restart times after crashes or shutdowns, which impacts data availability and business applications.

Innovation Solution

The proposed solution involves processing the transaction log backward, only writing changes from committed transactions and skipping incomplete transactions, allowing for a reverse chronological recovery of database rows to reinstate the final state of data efficiently.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If all log records are replayed from the transaction log to recover in-memory database data, then data completeness is ensured, but recovery time becomes excessively long

Engineering Contradiction:

Improvedata completenessVSAvoidrecovery time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent inverts the conventional forward replay approach by processing transaction logs in reverse chronological order from the latest committed transaction backward. This allows the recovery process to start with the most recent state and selectively apply only necessary changes, dramatically reducing recovery time while maintaining data completeness through systematic verification of committed transactions.

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The patent extracts and processes only the essential committed transactions from the transaction log, separating them from incomplete or uncommitted transactions. By identifying and recovering only the committed portion of the log in reverse order, the system achieves fast recovery without sacrificing data integrity, as uncommitted transactions are naturally excluded.

Inventive Principle:

Principle #2Taking out (Extraction)

2Reliability

If the entire transaction log is processed during recovery, then all database changes are recovered, but memory resources are excessively consumed

Engineering Contradiction:

Improverecovery completenessVSAvoidmemory usage

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

Processing in reverse chronological order enables the system to load and process only the most recent committed transactions that actually changed the database state. Earlier transactions are skipped once their effects are already present in the current state, significantly reducing memory consumption while ensuring all necessary changes are recovered.

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The patent applies partial action by processing only the necessary subset of transaction log records - specifically, committed transactions in reverse order - rather than processing the entire log. This selective approach recovers all required data changes while consuming minimal memory resources by avoiding redundant processing of already-applied changes.

Inventive Principle:

Principle #16Partial or excessive action

3Reliability

If conventional forward replay of transaction logs is used, then data consistency is maintained, but restart time after crash is prolonged

Engineering Contradiction:

Improvedata consistencyVSAvoidrestart time

Core Design Contradiction:

ReliabilityVSDuration of action of moving object

Solution Approach 1:

The patent fundamentally inverts the recovery direction from forward replay to backward processing. By starting with the latest committed transaction and working backward, the system可以快速 identify and apply only the necessary changes to restore data consistency, reducing restart time from potentially hours to minutes while maintaining full data consistency through systematic verification.

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The system performs preliminary identification of committed transactions before actual recovery processing. By pre-scanning and marking committed transactions in the log, the recovery process can directly apply only those changes without re-evaluating each record, thereby maintaining data consistency while dramatically reducing the actual restart time.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11416350B2Recovery of in-memory databases from log records

Publication Date: 2022.08.16 SAP SE

AI summary

Data recovery in a database includes performing a backward scan of a transaction log that records transactions made on the database. The backward scan includes traversing log records starting from a latest-in-time log record toward the beginning of the transaction log. Each log record can be replayed to recover data for a database row associated with the log record. An earlier-in-time log record is skipped if the associate database row has been recovered by later-in-time log record.