Chained Metadata Tables for Database Crash Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face challenges in ensuring consistency and efficient management of metadata tables, particularly during crashes or power failures, which can lead to data corruption and recovery issues due to the lack of effective mechanisms for handling additions, deletions, and modifications of metadata tables.
Innovation Solution
A metadata table management scheme that creates a chain of metadata tables with next metadata table keys linking them, allowing for efficient recovery by storing preallocated metadata table keys and begin user keys in a manifest, enabling the system to locate and access metadata tables even after crashes, and allowing for additions, deletions, and splits of metadata tables to improve search and retrieval speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single metadata table is used to store all database metadata, then the structure is simple and easy to manage, but the system cannot efficiently handle crashes or power failures and may lead to data corruption
Solution Approach 1:
The patent divides the single metadata table into multiple chained metadata tables, where each table stores a portion of the metadata and contains a pointer to the next table. This segmentation allows the system to isolate corruption to individual tables while maintaining overall database consistency through the chain structure, resolving the contradiction between reliability and complexity.
2Productivity
If metadata tables are frequently added, deleted, or modified to improve search and retrieval speed, then the system becomes more flexible and efficient, but the risk of data corruption during crashes increases
Solution Approach 1:
The patent implements a manifest that pre-stores metadata table keys and begin user keys before operations are executed. This preliminary action allows the recovery logic to reconstruct the metadata table chain even if a crash occurs during dynamic operations, enabling flexible table management while maintaining data consistency through pre-prepared recovery information.
Solution Approach 2:
The patent employs recovery logic that continuously monitors the metadata table chain and can detect inconsistencies. When a crash or corruption is detected, the recovery logic uses the manifest information to reconstruct the proper chain state, providing feedback control that maintains reliability despite frequent modifications for improved productivity.
3Speed
If the system stores all metadata table information in memory for fast access, then search and retrieval speed improves, but the system consumes more energy and memory resources
Solution Approach 1:
The patent implements a hybrid storage approach where critical metadata table structure information (keys and pointers) is stored in the manifest for fast access, while actual metadata contents remain in the chained tables on storage. This local quality differentiation allows the system to optimize access speed for structural information while minimizing memory energy consumption for bulk data storage.
Data Source
AI summary
Provided is a method of database management including locating, with a recovery logic, a first metadata table using a beginning metadata table key, reading, by the recovery logic, the first metadata table, retrieving, with the recovery logic, a first next metadata table key of the first metadata table, locating, by the recovery logic, a second metadata table based on the first next metadata table key or based on a third next metadata table key of a third metadata table having a third metadata table range between a first metadata table range of the first metadata table and a second metadata table range of the second metadata table, reading, by the recovery logic, the second metadata table, determining, by the recovery logic, the second metadata table lacks valid keys in the second metadata table range, and making available, by the recovery logic, memory space associated with the second metadata table.