Incremental Database Dump Using Consistency Markers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database backup and replication processes are time and resource intensive, often locking data and preventing other processes until locks are released, as they typically involve copying entire databases or tables, which is inefficient for disaster recovery, data warehousing, and testing.
Innovation Solution
Implementing an initial full dump of a database table, tracking changes outside of transactions, and using partial dumps to create a data structure that updates only modified rows since the last dump, allowing for efficient instantiation of database table instances by sequencing full and partial dumps from earliest to most recent.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a full database dump is taken to ensure complete data replication, then data completeness is improved, but time consumption and resource usage increase significantly
Solution Approach 1:
The patent divides the database dump process into two segments: an initial full dump to establish baseline completeness, and subsequent incremental dumps to capture only changes. This segmentation allows the system to maintain data completeness while significantly reducing the time and resources required for repeated dumps by only transferring modified rows rather than the entire database.
Solution Approach 2:
The patent implements partial action by performing incremental dumps that capture only the portion of data that has changed since the last dump (identified through consistency markers). Instead of repeatedly dumping the entire database, the system applies partial action to transfer only necessary data, reducing time consumption while maintaining completeness through the combination of full and incremental dumps.
2Reliability
If a full database dump is taken to ensure complete data replication, then data completeness is improved, but resource consumption increases significantly
Solution Approach 1:
The patent segments the data transfer process into a one-time full dump followed by efficient incremental dumps. This segmentation reduces resource consumption for repeated operations by identifying and transferring only modified rows using consistency markers, while the initial full dump ensures complete data completeness is achieved.
Solution Approach 2:
The patent uses copying efficiently by creating a full copy initially, then copying only the changes (incremental dumps) in subsequent operations. This approach reduces resource consumption for repeated dumps while maintaining data completeness through the combination of the initial full copy and subsequent change copies.
3Reliability
If traditional dump processes are used to replicate databases, then data replication is achieved, but data locking occurs preventing other processes
Solution Approach 1:
The patent applies preliminary action by establishing consistency markers before dumps are taken. These markers are set in advance to identify the state of the database at specific points in time, allowing incremental dumps to capture changes without requiring locks on the data. This preliminary marking enables continuous operation while maintaining replication reliability.
4Productivity
If incremental dumps are used to reduce dump time, then time efficiency is improved, but implementation complexity increases
Solution Approach 1:
The patent reduces complexity by performing preliminary action through the establishment of consistency markers. These markers are set in advance to identify database states, enabling incremental dumps to efficiently capture changes without complex real-time tracking or locking mechanisms. This preliminary marking simplifies the incremental dump process while maintaining productivity.
Data Source
AI summary
Various embodiments may include taking an initial dump of a database table and tracking changes to the database table, with regard to but outside of transactions, since a most recent full or partial dump has been taken. A partial dump, in such embodiments, may be taken since the most recent full or partial dump and used to create a data structure which may be used to update only those database table rows modified since a last full or partial dump. As a result, the dumps of a database table will include a single full dump and may include one or more partial dumps. These full and partial dumps may then be instantiated to create or update an instance of the database table. In some embodiments, dumps may be taken directly to a mirrored instance of a database table.