Mixed Metadata and Non-Metadata Logging for Database Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database systems face challenges in efficiently managing metadata and non-metadata logging, leading to increased complexity and overhead, particularly during high workloads and disaster recovery scenarios, where maintaining a consistent state between primary and backup systems is crucial for high availability and load balancing.
Innovation Solution
The implementation of a mixed logging method that separates metadata and non-metadata logging, using logical logs for metadata and physical logs for non-metadata, allowing for efficient data communication and recovery processes, enabling quick restart and load balancing between primary and backup systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a complete copy of the database is stored at both datacenters with synchronization mechanism, then system availability is improved, but device complexity and synchronization overhead increase
Solution Approach 1:
The patent segments the database into metadata objects and non-metadata data objects, and further divides logging into metadata logs and non-metadata logs. This segmentation allows independent recovery processes for different data types, reducing the complexity of synchronizing the entire database while maintaining system availability.
Solution Approach 2:
The patent extracts metadata logging from the general database logging mechanism, creating a separate metadata log system. This extraction allows metadata to be recovered independently from non-metadata data, simplifying the synchronization mechanism and reducing overhead while maintaining reliability.
2Productivity
If traditional logging methods are used for both metadata and non-metadata, then implementation is simpler, but logging efficiency and recovery speed decrease during high workloads
Solution Approach 1:
The logging mechanism is segmented into metadata logs and non-metadata logs, allowing parallel processing and independent optimization of each log type. This segmentation improves logging efficiency during high workloads by enabling simultaneous recovery operations without the bottlenecks of unified logging.
Solution Approach 2:
The patent introduces an intermediary layer that separates metadata logging from non-metadata logging, with distinct log structures and recovery processes. This intermediary approach improves efficiency by allowing specialized handling of each log type while maintaining overall system coherence.
3Loss of time
If metadata and non-metadata are recovered together in a unified process, then implementation is easier, but recovery time increases during disaster recovery scenarios
Solution Approach 1:
The recovery process is segmented into independent metadata recovery and non-metadata recovery operations. This allows parallel execution of recovery tasks, significantly reducing total recovery time during disaster recovery scenarios while managing complexity through modular recovery procedures.
Solution Approach 2:
The patent implements preliminary separation of metadata and non-metadata during normal operations, preparing them for independent recovery. This preliminary action enables faster disaster recovery by having pre-separated data ready for parallel restoration, reducing the time loss during critical recovery scenarios.
4Reliability
If physical logging is used for all data, then data integrity is ensured, but storage overhead and I/O operations increase
Solution Approach 1:
The patent segments logging into metadata logs (logical) and non-metadata logs (physical), allowing different logging strategies for different data types. This segmentation maintains data integrity for critical metadata through logical logging while reducing I/O overhead for bulk non-metadata data through physical logging.
Solution Approach 2:
Different logging qualities are applied locally to different data types: metadata receives detailed logical logging for integrity, while non-metadata uses compact physical logging to reduce I/O. This local quality approach maintains necessary data integrity while minimizing energy loss from I/O operations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Restoring a database image of a secondary database system comprises initializing the database image from a recovery image that includes metadata logs and non-metadata logs, of which at least one metadata log includes an undo log of non-committed data that is applied to the initialized database image. A first pagelist and first indexes are then rebuilt based on first data segments of the database image and the corresponding metadata objects. The database image is then updated by replaying the metadata redo log generated from the undo log, the metadata logs, and the non-metadata logs. Then, a second pagelist and second indexes for the database system are rebuilt based on second data segments of the database image and the corresponding non-metadata objects, while each of the first data segments is statically separated from the second data segments in physical memory space. Related apparatus, systems, techniques and articles are also described.