Database Migration via Secondary Storage Snapshots
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database migration processes are hindered by significant downtime due to the large size of database files, which can take several hours to migrate, causing inconsistencies and impacting user access and performance.
Innovation Solution
A secondary storage system creates a backup snapshot of the database using a tree data structure, allowing for instantaneous or near-instantaneous access by cloning a view of the snapshot, and migrates database content in the background while maintaining access, using a priority-based approach to optimize IOPS performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If database files are migrated using traditional methods, then data consistency is maintained, but migration time becomes unacceptably long and database downtime increases
Solution Approach 1:
The system creates a snapshot of the database files before migration begins. This preliminary action captures the state of the data at a specific point in time, allowing the migration to proceed from this fixed baseline while the original database continues to accept writes. The snapshot serves as a consistent starting point that prevents data inconsistency without requiring the database to be quiesced during the entire migration process.
Solution Approach 2:
The system creates a copy of the database files through snapshotting and migrates this copy rather than the live files. This copying mechanism allows the original database to remain operational while the snapshot copy is migrated to the new location. The copied data provides a consistent baseline for migration while eliminating the need to stop the source database.
2Quantity of substance
If database files are migrated to reduce storage costs, then storage efficiency improves, but IOPS performance deteriorates due to remote access
Solution Approach 1:
The system implements a hybrid storage architecture where hot data (frequently accessed database files) is cached locally on high-performance storage at the database server, while cold data (less frequently accessed files) is stored remotely on cost-effective storage systems. This local quality differentiation ensures that critical IOPS performance requirements are met for active operations while achieving storage cost reduction through remote archiving of less critical data.
3Productivity
If database files are kept local to maintain performance, then IOPS performance is maintained, but storage scalability and cost-effectiveness are limited
Solution Approach 1:
The system transitions from a single-dimension local storage model to a multi-dimensional storage architecture that combines local high-performance storage with remote scalable storage. By adding the remote storage dimension, the system achieves both goals: local storage maintains IOPS performance for critical operations, while remote storage provides unlimited scalability and cost-effectiveness for expanding database capacity.
Data Source
AI summary
A request to migrate from a secondary storage system to a server a requested version of database content of a database backed up at the secondary storage system is received. A new version of the database content at the secondary storage system is instantiated. Functionality of the database is maintained including by allowing a database application hosted on the server to access the database content from the secondary storage system using the instantiated new version of the database content while migrating the requested version of database content from the secondary storage system to the server.