Range-Partitioned Table Archiving for Source-Target Coherence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional database management systems (DBMS) struggle to support both operational and analytical workloads simultaneously, leading to data coherence issues when multiple DBMSs are involved, as they specialize in either operational or analytical functions.
Innovation Solution
A method for archiving data in a data analysis system that includes partitioning source tables by ranges of attributes, synchronizing data between a source and target database system, and using replication technology to maintain data coherence, allowing seamless archiving and optimizing storage and processing resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is archived by removing partitions from the source table, then storage space is freed and performance is improved, but data coherence between source and target systems may be compromised
Solution Approach 1:
The system performs preliminary synchronization of the source partition to the target partition before removing data from the source. This ensures that archived data is safely stored in the target system before the source partition is deleted, maintaining data coherence and preventing data loss.
Solution Approach 2:
The invention creates a copy of the source partition data in the target partition before removal. The replication mechanism copies data from the source table partition to the target table partition, ensuring data coherence is maintained while enabling safe archiving and removal from the source system.
2Reliability
If data synchronization is performed before archiving, then data coherence is maintained, but processing time and system resources are consumed
Solution Approach 1:
The system divides the archiving process into distinct phases: synchronization phase and removal phase. By segmenting the operation, the system can synchronize only the specific partition being archived rather than the entire database, reducing overall processing time while maintaining data coherence for the archived data.
Solution Approach 2:
The system performs synchronization only for the specific partition that needs to be archived, rather than synchronizing the entire database. This partial action approach maintains data coherence for the archived partition while minimizing the time and resources consumed by the synchronization process.
3Manufacturing precision
If source partitions are locked during archiving, then data integrity is ensured, but system availability and write operations are reduced
Solution Approach 1:
The system locks only the specific partition being archived rather than the entire table or database. This segmented locking approach ensures data integrity for the archived partition while allowing other partitions to remain accessible for read and write operations, maintaining system availability.
Solution Approach 2:
The lock is applied only partially to the specific partition being archived rather than to the entire database system. This allows other parts of the system to continue operating normally, maintaining ease of operation and system availability while still ensuring data integrity for the archiving operation.
Data Source
AI summary
An embodiment for archiving data in a data analysis system. The embodiment may receive a request to archive a source partition of a source table of source database system of the data analysis system. The embodiment may determine whether the source table is partitioned into partitions by ranges of an attribute of the source table. The embodiment may, in response to determining that the source table is partitioned by range, lock the source partition to prevent changes of the source partition. The embodiment may determine whether the source partition is synchronized with a corresponding partition in the target table. In response to determining that the source partition is synchronized with the corresponding partition in the target table, the embodiment may remove data in the source partition from the source table.