Secondary Storage System for Database Migration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database migration processes are lengthy and result in significant downtime, as existing methods require quiescing the database to ensure consistency, leading to unfulfilled database requests and unacceptable downtime due to the large size of database files, typically taking hours to migrate 10 TB of data.
Innovation Solution
Implementing a secondary storage system that uses a tree data structure to manage backup snapshots, allowing incremental backups and migration of database content in the background, enabling continuous database usage while migrating data to an upgraded version, and providing options for quiescing based on downtime thresholds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the database is quiesced to ensure consistency during migration, then data consistency is improved, but downtime increases significantly
Solution Approach 1:
The patent performs preliminary actions by creating backup snapshots of the database files before migration begins. These snapshots capture the database state at specific points in time, allowing the system to restore consistency later without requiring extended quiescence. The backup agent proactively prepares these snapshots while the database is still operational.
Solution Approach 2:
The patent introduces an intermediary backup snapshot mechanism that mediates between the source and target database systems. Instead of directly migrating live data which would require quiescence, the backup snapshot acts as an intermediary artifact that can be copied and applied later, enabling migration without stopping the database.
2Productivity
If the database files are copied directly from source to target, then migration speed is improved, but data consistency deteriorates
Solution Approach 1:
The system performs preliminary backup of database files to a snapshot before migration. This preliminary action creates a consistent baseline that can be rapidly copied to the target system, maintaining both speed and consistency. The snapshot serves as a pre-prepared consistent state that eliminates the need for slow synchronous replication.
Solution Approach 2:
The patent uses copying of backup snapshots instead of direct live data replication. The backup snapshot, which captures a consistent state of the database, is copied to the target system. This copying approach is much faster than maintaining real-time consistency during migration, while still ensuring data integrity through the snapshot mechanism.
3Loss of time
If incremental backups are performed during migration, then downtime is reduced, but system complexity increases
Solution Approach 1:
The patent segments the migration process into distinct phases: initial backup snapshot creation, incremental backup updates, and final synchronization. Each phase handles a portion of the data consistency requirement. The backup agent is segmented to perform different types of backups (full, incremental) at different stages, reducing overall downtime while managing complexity through structured segmentation.
Solution Approach 2:
The system maintains continuity of useful action by performing incremental backups during the migration process without stopping the database. The backup agent continues to capture changes incrementally while the database remains operational, ensuring that migration proceeds continuously rather than in interrupted batches, thereby reducing total downtime.
Data Source
AI summary
A portion of contents of a database is received from a first server. The received contents of the database is stored in a secondary storage system that tracks changes between different backup versions of contents of the database. A request to migrate the contents of the database to a second server is received. A version of contents of the database is provided to the second server using the secondary storage system. The secondary storage system is configured to determine an amount of changes to the database content from one of the versions of the database content provided to the second server and the amount of changes is utilized in determining whether to quiesce the database hosted on the first server.