Sparse Database Snapshots for Storage Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Database copying for snapshots is resource-intensive and storage-space-consuming, and can cause instability due to incompatible changes during the software development lifecycle, as it requires deep copying of all data and maintaining multiple copies of large databases.
Innovation Solution
Creating a sparse database snapshot that initially contains only metadata, with data files that are sparse files backed by the parent database, allowing for efficient read and write operations without the need for deep copying, enabling daisy chaining and reducing storage requirements through thin provisioning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a database snapshot is created by deep copying all data, then data isolation and stability are improved, but storage space consumption and time required increase significantly
Solution Approach 1:
The database snapshot is segmented into a sparse database structure that only stores modified blocks, rather than copying the entire database. The sparse database contains only the differences (delta) between the original and snapshot, significantly reducing storage requirements while maintaining data isolation through the separation of original and snapshot data paths.
Solution Approach 2:
Instead of deep copying all data, the system creates a sparse copy that references the original database for unchanged blocks. Only modified blocks are copied to the sparse database, achieving a lightweight copy that maintains isolation without consuming full storage space.
2Reliability
If a database snapshot is created by deep copying all data, then data isolation is improved, but the time required for snapshot creation increases
Solution Approach 1:
The snapshot creation process is segmented into identifying modified blocks rather than copying all blocks. By segmenting the database into changed and unchanged portions, the system only processes and copies the necessary modified segments, dramatically reducing snapshot creation time while maintaining isolation.
Solution Approach 2:
The system performs preliminary identification of modified blocks before actual copying. By pre-determining which blocks need to be copied based on change detection, the system avoids unnecessary copying operations and reduces overall snapshot creation time.
3Reliability
If multiple database copies are maintained for different development stages, then data isolation for each stage is improved, but resource stress and instability increase
Solution Approach 1:
Multiple sparse databases are maintained to represent different development stages, each containing only the differences from the previous stage. This segmentation allows data isolation between stages while minimizing the total data volume that needs to be managed, reducing resource stress compared to maintaining multiple full copies.
Solution Approach 2:
The sparse databases are nested in a hierarchical structure where each sparse database references the previous stage's database. This nesting allows multiple development stages to be maintained with minimal duplication, as each stage only stores its differences relative to the previous stage, reducing overall resource consumption.
Data Source
AI summary
Techniques herein are for creating a database snapshot by creating a sparse database. A method involves receiving a creation request to create a sparse database. The creation request has an identity of a parent database. The creation request is processed to create a sparse database. The sparse database has the identity of the parent database. The sparse database does not contain data copied from the parent database. A write request to write data into the sparse database is received. The write request is processed by writing the data into the sparse database. The parent database does not receive the data.