Logical Database Copy via Partitioned Extent Chunks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methodologies for copying data from a source database to target databases in cloud-based computing are slow and inefficient, leading to excessive storage requirements, fragmentation, and prolonged downtime due to the need to copy all data, even if only a portion is necessary.
Innovation Solution
A logical copy method that partitions data into extent chunks and uses multi-threading to copy only necessary data, allowing for parallel processing and reducing the amount of data transferred, thereby minimizing storage needs and avoiding fragmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional physical copy methodologies are used to replicate data from source database to target databases, then data replication can be achieved, but the copying process is slow and inefficient
Solution Approach 1:
The patent segments the data copying process into multiple parallel operations using multi-threading. Instead of copying data sequentially, the system divides the source database into multiple partitions and assigns different threads to copy different partitions simultaneously, dramatically improving data copying speed and reducing migration time.
Solution Approach 2:
The patent performs preliminary actions by creating partition information and identifying necessary data before the actual copying process. The system pre-processes the source database to determine which partitions and data blocks need to be copied, allowing the copying operation to proceed efficiently without unnecessary data transfer.
2Reliability
If all data is copied from source database to target databases, then complete data replication is achieved, but storage requirements increase excessively
Solution Approach 1:
The patent extracts only the necessary data from the source database based on partition information and copying requirements. Instead of copying all data, the system identifies and copies only the specific partitions and data blocks that are needed, significantly reducing storage requirements while maintaining data completeness for the required datasets.
Solution Approach 2:
The patent applies partial action by copying only the necessary portions of data rather than the entire database. The system determines the minimal required data set based on partition specifications and copies only those portions, avoiding excessive data transfer and storage consumption while ensuring all necessary data is replicated.
3Productivity
If conventional copying methods are used, then data replication is achieved, but data fragmentation occurs in target databases
Solution Approach 1:
The patent uses segmentation to divide data into organized partitions with defined structures. By copying data in structured partition units rather than as unorganized data blocks, the system maintains data structure integrity and prevents fragmentation in the target database while still achieving efficient replication.
Solution Approach 2:
The patent performs preliminary organization of partition information before copying, establishing the target data structure in advance. This pre-structuring ensures that data is copied into properly organized partitions from the beginning, maintaining structural integrity and preventing fragmentation issues that would occur with ad-hoc copying methods.
4Reliability
If all data is copied from source database, then complete replication is achieved, but downtime is prolonged
Solution Approach 1:
The patent segments the data replication process into parallel partition copying operations. Multiple partitions are copied simultaneously through multi-threading, dramatically reducing the total time required to replicate data while ensuring complete replication of all necessary partitions. This parallel processing approach minimizes downtime while maintaining replication completeness.
Solution Approach 2:
The patent applies partial action by replicating only the necessary partitions and data blocks rather than copying the entire database. This selective replication reduces the total volume of data that needs to be transferred and processed, thereby reducing downtime while ensuring all required data is replicated for system availability.
Data Source
AI summary
A system and method are provided for logically copying data from a source database to a first target database and a second target database. Based on table partition information, the source database is queried to collect partition metadata information for a first set of partitions and a second set of partitions. A first set of the partition metadata information for the first set of partitions and a second set of the partition metadata information for the second set of partitions can be used to create at least one extent chunk for each partition of a table. The source database can be queried, based on a first set of extent chunks and a second set of extent chunks, for a first set of data to be written to the first target database and a second set of data from the source database to be written to the second target database.