Incremental Data Transfer in Database Management Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database management systems are inefficient in data transfer, as they often require exporting all data from a table, leading to the transfer of large volumes of repetitive data, which slows down processing speed and increases workload.
Innovation Solution
Implementing incremental data transfer in database management systems by identifying eligible tables, marking changed data with timestamps, and transferring only committed and updated rows, along with a transfer history mechanism to track and manage data changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all data is exported from the database table, then complete data transfer is achieved, but data transfer volume increases and processing speed decreases
Solution Approach 1:
The patent extracts only the necessary changed data from the database table using an incremental data transfer mechanism. By identifying and transferring only modified rows since the last export, it removes unnecessary data from the transfer process while maintaining data completeness for changed records.
Solution Approach 2:
The patent segments the data transfer process into identifying changed data, selecting relevant columns, and transferring only necessary information. This segmentation allows the system to transfer data in manageable portions rather than as a single large operation, improving transfer efficiency.
2Reliability
If all data is exported from the database table, then data completeness is ensured, but workload at the receiver increases
Solution Approach 1:
The patent extracts only the essential changed data and relevant columns at the source system before transfer. This pre-filtering reduces the workload at the receiver by eliminating unnecessary data processing, while still ensuring completeness of the transferred changed records.
Solution Approach 2:
The patent performs preliminary data identification and selection at the source system before transfer occurs. By pre-identifying changed rows and selecting relevant columns in advance, it reduces the processing burden on the receiver system while maintaining data completeness.
3Productivity
If incremental data transfer is implemented, then data transfer efficiency improves, but system complexity increases
Solution Approach 1:
The patent introduces an intermediary mechanism within the database management system that tracks and identifies changed data using timestamps or version information. This intermediary layer manages the complexity of incremental identification while maintaining efficient data transfer, without requiring complex external systems.
Data Source
AI summary
System, method, computer program product embodiments and combinations and sub-combinations thereof for incremental data transfer in a database management system (DBMS) are provided. In an embodiment, a database table enabled for partial export from a database management system is identified. Data within the database table to be exported is also identified, and the identified data is transferred in a chosen format from the database table to an export location.