Selective Table Replication for Stream Data Integrity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional table replication methods in network-based database systems do not adequately account for stream replication, leading to loss of valuable information such as table versions and partitions.
Innovation Solution
The described techniques selectively replicate select partitions and table versions needed for stream replication, using existing table partitions to enable stream replication without significant additional storage and computational overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional table replication methods are used, then replication process is simple, but stream replication information (table versions and partitions) is lost
Solution Approach 1:
The patent segments the table into multiple versions and partitions, selectively replicating only the necessary segments (table versions and partitions) required for stream replication rather than replicating the entire table. This segmentation enables preservation of stream replication information while reducing replication complexity.
Solution Approach 2:
The patent performs preliminary actions by pre-identifying and pre-selecting which table versions and partitions need to be replicated for stream replication before the actual replication process. This preliminary selection mechanism ensures that stream replication information is preserved without adding significant complexity to the overall replication process.
2Reliability
If full table replication is performed, then data availability is ensured, but storage overhead and computational cost increase
Solution Approach 1:
The patent extracts only the essential components (specific table versions and partitions) needed for stream replication from the full table, rather than replicating the entire table. This extraction approach maintains data availability for stream replication while significantly reducing storage overhead and computational resources required.
Solution Approach 2:
The patent applies partial action by replicating only the necessary portion of the table (select table versions and partitions) rather than the complete table. This partial replication suffices for stream replication needs, reducing storage overhead while maintaining adequate data availability for the intended purpose.
3Quantity of substance
If selective partition replication is implemented, then storage overhead is reduced, but complexity in determining which partitions to replicate increases
Solution Approach 1:
The patent implements self-service by enabling the database system to automatically determine and identify which table versions and partitions need to be replicated for stream replication, without requiring manual intervention or complex external selection mechanisms. The system uses its own metadata and indexing capabilities to self-determine the replication scope, reducing storage overhead while keeping the selection mechanism manageable.
Data Source
AI summary
Techniques described herein can enable stream replication. A first deployment can store a table including one or more streams. The techniques described herein can be used to replicate the table at a second deployment while replicating the one or more streams associated with the table. Select prior table versions and partitions in the table are copied to the second deployment to enable stream replication.