Logical Log Records for In-Memory Database Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional database recovery systems are inadequate for in-memory database systems, as they rely on physical logging that is inefficient and not adaptable for high availability, parallelism, and integration with other systems, lacking versatility and mirroring capabilities.
Innovation Solution
Implementing a logical logging system that logs transactions without physical location information, allowing for parallel processing, reduced dependency on physical checkpoints, and integration with existing two-phase commit protocols, enabling efficient recovery and mirroring across multiple nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If physical logging is used in in-memory database systems, then recovery can be performed, but the system lacks versatility and adaptability for high availability and parallelism
Solution Approach 1:
The patent changes the fundamental parameter of logging from physical location-based to logical transaction-based. Instead of logging operations with reference to specific disk pages and physical addresses, the system logs transactions with logical identifiers and timestamps. This parameter change enables the same logging mechanism to serve multiple purposes including crash recovery, high availability, and parallel processing without the constraints of physical location dependency.
Solution Approach 2:
The logical logging system is designed to be universal, serving multiple functions simultaneously. The same log records support crash recovery, high availability operations, and integration with two-phase commit protocols. This multi-functionality is achieved by recording transactions in a location-independent manner that can be replayed and validated across different system states and configurations.
2Productivity
If physical logging with location information is implemented, then detailed operation tracking is achieved, but parallelism and integration with other systems are limited
Solution Approach 1:
The patent extracts physical location information from the logging process. By removing page identifiers, row positions, and storage addresses from log records, the system eliminates the coupling between logging and physical storage architecture. This extraction enables parallel processing of log records without conflicts and simplifies integration with other database systems that may have different physical storage organizations.
3Reliability
If conventional physical checkpointing is used, then data consistency is maintained, but resource usage increases and versatility decreases
Solution Approach 1:
The patent uses logical copying instead of physical checkpointing. Rather than creating actual copies of data pages at checkpoint moments, the system uses logical log records that can represent the state of data at any point in time. These log records are much smaller and require minimal resources to store and process, yet provide the same consistency guarantees as traditional checkpointing mechanisms.
Data Source
AI summary
The subject disclosure relates to a database recovery technique that implements various aspects of transaction logging to optimize database recovery performance. Transactions are logged logically with no reference to physical location, which enables logging to be performed via multiple independent log streams in parallel. A set of log streams can further be adjusted to conform to a local configuration of a mirror or secondary node in order to increase mirroring flexibility. Additionally, individual transactions or groups of transactions are recorded using a single log record, which contains timestamp information to enable database recovery without reference to physical checkpoint files. Further, techniques are provided herein for processing transactions without Write Ahead Logging or hardening of undo information. As further described herein, a transaction can be committed via a set of hierarchical stages, which in turn can facilitate integration of an in-memory database system with one or more external database systems.