Database Recovery via DMS Agent Bitmap Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data management and storage systems face challenges in minimizing database downtime during recovery processes, leading to inevitable losses and poor user experiences due to the gradual transfer of data blocks or files from backup sources to destination systems.
Innovation Solution
A system that uses a bitmap to track the transfer of data blocks or files, allowing for concurrent transfer and directing read and write operations between local and source storage systems to minimize downtime, with the DMS agent intercepting commands and managing the transfer process in kernel mode.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data blocks are transferred gradually from source storage to local storage during recovery, then data integrity is maintained, but database downtime increases
Solution Approach 1:
The system performs preliminary actions by pre-transferring data blocks from source storage to local storage before they are actually needed for database operations. The DMS agent predicts which data blocks will be needed and initiates their transfer in advance, so that when read commands arrive, the data is already available locally, eliminating downtime while maintaining integrity through controlled transfer
Solution Approach 2:
The system dynamically adjusts the data transfer process based on real-time conditions. The DMS agent monitors database operations, identifies access patterns, and adaptively prioritizes which data blocks to transfer first. This dynamic approach allows the system to optimize transfer timing and sequencing, ensuring critical data is available while minimizing overall downtime
2Speed
If all data blocks are transferred to local storage before recovery operations begin, then read operations can be served locally, but transfer time and resource consumption increase
Solution Approach 1:
The system applies partial action by transferring only the necessary subset of data blocks to local storage rather than all data blocks. The DMS agent identifies and transfers only those data blocks that are predicted to be needed for upcoming database operations, leaving less frequently accessed blocks on source storage. This selective approach provides fast local access for critical data while avoiding the overhead of transferring unnecessary data
Solution Approach 2:
The data transfer process is segmented into priority levels and batches. The DMS agent divides data blocks into segments based on their predicted importance and timing requirements, transferring high-priority blocks first while lower-priority blocks remain on source storage. This segmentation allows the system to achieve fast local access for critical operations without the cost of transferring the entire dataset
Data Source
AI summary
An example method of restoring a database includes obtaining information about backup data of a database from a source storage separate from a compute infrastructure. The information includes a list of data blocks of a file, transferring the data blocks on the list from the source storage to a local storage on the compute infrastructure, and tracking which data blocks of the file have been transferred from the source storage to the local storage concurrently when transferring the data blocks.