Dynamic Data Copy Utility for Object-Relational Databases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data copying platforms in object-relational database environments are limited in functionality, requiring identical table formats, high-level privileges, and lacking intelligence to handle structural differences and performance optimization, leading to inefficient data copy operations that impact concurrent users and network performance.
Innovation Solution
A high-performance data copy utility that dynamically copies data by defining performance boundaries such as data quantity and parallel processes, allowing for interactive progress updates, and enabling data copying without high-level privileges, with features like table exclusion, data filtering, and automatic dependency management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If current copying platforms override all existing data with copied data to create exact replicas, then data copy completeness is improved, but flexibility to modify database structure and filter data deteriorates
Solution Approach 1:
The copying process is segmented into multiple independent components: structure analysis, data filtering, transformation rules application, and selective copying. This allows the system to maintain exact replica capability while simultaneously enabling structural modifications and data filtering through configurable parameters.
Solution Approach 2:
The copying platform transitions from a static exact-replica-only approach to a dynamic system where copying behavior can be adjusted in real-time. Users can configure transformation rules, filter conditions, and structural modifications that are applied during the copy process, making the system adaptable to different requirements.
2Ease of operation
If current copying platforms require identical table formats to perform copying, then copying simplicity is improved, but adaptability to handle structural changes deteriorates
Solution Approach 1:
An intermediary transformation layer is introduced between the source and target databases. This layer analyzes structural differences, applies transformation rules, and handles format conversions automatically. Users can define mapping rules and transformation logic that bridge structural gaps, enabling copying between different table formats while maintaining operational simplicity.
Solution Approach 2:
The system dynamically adjusts copying parameters based on detected structural differences. When source and target table formats differ, the system automatically modifies transformation parameters, data types, and mapping relationships to accommodate structural changes, eliminating the requirement for identical table formats.
3Adaptability or versatility
If current copying platforms require heightened account privileges such as DBA or DDL, then ability to perform structural changes is improved, but ease of operation and security deteriorates
Solution Approach 1:
The copying platform provides self-service capabilities that allow users to perform structural analysis, transformation rule definition, and data copying without requiring DBA or DDL privileges. The system automatically handles structure reconciliation and applies transformations using standard data manipulation language (DML) operations that can be executed with lower-level account privileges.
4Ease of manufacture
If current copying platforms perform copying without performance optimization, then implementation simplicity is improved, but copying speed and impact on concurrent users deteriorates
Solution Approach 1:
The copying process implements periodic action through batch processing and commit point mechanisms. Data is copied in controlled batches with periodic commits to the target database, allowing concurrent users to continue operations between batches. This periodic approach optimizes copying speed while minimizing impact on database performance and user operations.
Solution Approach 2:
The system dynamically adjusts copying performance parameters based on real-time database conditions, network latency, and concurrent user activity. Copying speed, batch size, and parallel process utilization are dynamically optimized to maximize productivity while considering the impact on concurrent users and network resources.
Data Source
AI summary
Embodiments of the invention relate to systems, methods, and computer program products for a high performance data copy utility that is operable to dynamically copy data from a source database to a target database in an object-relational database environment. High performance is realized by defining, prior to copying the data, performance boundaries of the source and target environment, such as the quantity of data (i.e., commit points) and quantity of parallel processes (i.e., parallel threads) applicable to a copy job. Moreover, the copying schema of the present invention allows for data to be copied from one database to another database without the need higher level privileges, such as database administrator (DBA) or data definition language (DDL), which would otherwise require changing the structure of the database. In this regard, embodiments of the present invention rely solely on lower-level data manipulation language (DML) privileges configured for inserting, updating and deleting data structures.