Database Copy Execution via Metadata Evaluation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face challenges in efficiently managing data replication between different client sections while ensuring data consistency and minimizing downtime, particularly when copying data from one client environment to another, as they often require locking data and restricting access to prevent discrepancies and interference during the copying process.
Innovation Solution
The system evaluates statistical and update metadata to determine relevant tables for copying, performs iterative evaluations to define operations based on table categories and copy profiles, and executes a two-phase process of deletion and copying within the database layer without involving the application layer, allowing for parallel processing and reduced locking time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data copying is performed by locking source and target sections to prevent discrepancies, then data consistency is improved, but downtime and loss of time increase
Solution Approach 1:
The system performs preliminary evaluation of statistical metadata and update metadata before executing the copy operation. This allows the system to identify only the necessary tables and data records that require copying, preparing a targeted plan that minimizes the scope of locking and reduces downtime while ensuring data consistency for the relevant data sets.
Solution Approach 2:
The copy operation is divided into multiple phases: evaluation phase (analyzing metadata), execution phase (copying only identified tables), and commit phase. By segmenting the operation, the system locks only the specific tables being copied rather than the entire database section, reducing downtime while maintaining consistency for the affected data.
2Reliability
If all tables are copied to ensure complete data replication, then completeness of data is improved, but productivity and processing time worsen
Solution Approach 1:
The system performs partial copying by evaluating statistical metadata and update metadata to identify only the necessary subset of tables that contain updated data records. This partial action approach copies only what is needed rather than all tables, improving productivity while ensuring completeness of the relevant data through targeted replication.
Solution Approach 2:
The system extracts and analyzes update metadata to identify which tables have been modified since the last copy operation. By taking out only the updated tables for copying, the system ensures completeness of replicated data while significantly reducing processing time and resource consumption compared to copying all tables.
3Ease of operation
If the application layer is involved in the copying process to manage data operations, then ease of operation is improved, but device complexity and processing time increase
Solution Approach 1:
The copying operation is extracted from the application layer and executed directly at the database layer. The system evaluates metadata and performs copy operations within the database management system, eliminating the need for application layer involvement. This reduces device complexity by removing intermediate layers while maintaining ease of operation through automated database-managed processes.
Solution Approach 2:
The database system performs the copy operation autonomously by evaluating its own metadata and executing the copying process without external application layer intervention. This self-service approach simplifies the system architecture by eliminating unnecessary layers while ensuring proper data operation management through the database's native capabilities.
Data Source
AI summary
The present disclosure relates to computer-implemented methods, software, and systems for managing data replication between different source sections and target sections in response to received copy instructions associated with copy profiles. In response to evaluating statistical metadata identifying whether data records in relation to at least one client are included for a table from a first set of tables, a first subset of tables from the first set of tables is determined. In response to evaluating update metadata defining latest updates of tables from the first subset of tables, a second subset of tables from the first subset of tables is determined that defines tables that include updated data records relevant for copying. The second subset tables are iteratively evaluated to define corresponding operations to be performed for tables at the target section and at the source section in the database in relation to the requested copy operation.