Logical Clone Replication Using Table Differences and Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Replicating entire databases for disaster recovery or other scenarios significantly increases storage costs due to the need for physical data copying.
Innovation Solution
Implementing logical clone replication using a network-based data system that applies difference operations to refresh data instead of copying entire tables, and utilizing partition files with deduplication to reduce data transfer and storage resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If physical data copying is used to replicate entire databases for disaster recovery, then data availability and integrity are ensured, but storage costs and data transfer resources significantly increase
Solution Approach 1:
The patent applies logical copying instead of physical copying. Instead of copying entire tables physically, the system creates logical representations that reference the same physical data storage locations. This allows multiple users or systems to access the same data without duplicating the actual data storage, thereby reducing storage costs while maintaining data availability.
Solution Approach 2:
The patent introduces a logical representation as an intermediary between the physical data storage and the users/systems accessing the data. This logical layer acts as a mediator that provides data availability and integrity guarantees without requiring physical duplication of the underlying data, thus resolving the contradiction between reliability and storage costs.
2Reliability
If physical data copying is used to replicate entire databases, then data integrity is maintained, but data transfer resources and time significantly increase
Solution Approach 1:
The system copies logical representations rather than physical data. The logical representations are lightweight metadata structures that point to the actual data locations, enabling rapid creation of replicas without the time-consuming process of transferring entire datasets. Data integrity is maintained through the logical links to the original data.
Solution Approach 2:
The patent segments the data access model into physical data storage and logical representations. This segmentation allows the heavy data to remain in its original location while lightweight logical pointers are replicated, dramatically reducing transfer time while preserving integrity through the segmented reference structure.
3Reliability
If entire tables are copied for replication, then data completeness is ensured, but storage resources and costs significantly increase
Solution Approach 1:
The patent copies only the logical representation structure rather than the actual table data. The logical representation contains metadata about the data (such as schema information and location references) but not the data itself. This enables complete data access while using minimal storage resources, as multiple logical representations can reference the same physical data.
Solution Approach 2:
The logical representation serves multiple functions simultaneously: it provides data completeness by referencing all data locations, enables replication without copying data, reduces storage requirements, and maintains data integrity through reference links. This multi-functionality resolves the contradiction by making a single structure serve several purposes.
Data Source
AI summary
Data replication can be used to copy database data from a primary deployment to a secondary deployment in a network-based data system. Logical representation of the clone tables in the secondary deployment can be used to reduce data transfer and storage costs. In response to a refresh request, the data system may clone from existing tables stored in the secondary deployment by applying a difference operation on the existing tables instead of copying entire cloned tables for each refresh request.