Unified Data Refresh Stream in Replication Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing replication systems face challenges due to distinct data refresh and replication paths, leading to complexity and maintenance costs, as well as inefficiencies in handling refresh and change data, particularly with the internal refresh approach.
Innovation Solution
The introduction of a refresh stream that allows refresh data to flow through data record handling routines similar to change data records, enabling consistent behavior and leveraging existing data handling routines for both refresh and replication data, and allowing previously unsupported data sources to be refreshed while optimizing data access methods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If an internal refresh option is provided where the replication product accesses source data and sends it to the target system, then user convenience is improved, but device complexity increases due to the need for separate refresh paths and data handling routines
Solution Approach 1:
The patent merges the refresh path and change data path into a single unified path. Refresh data is now sent through the same capture service and data handling routines that process change data, eliminating the need for separate refresh infrastructure and reducing system complexity while maintaining user convenience
2Reliability
If separate data refresh and replication paths are maintained, then data handling consistency is preserved, but maintenance costs and system complexity increase
Solution Approach 1:
The capture service and data handling routines are designed to handle both refresh data and change data through the same universal interface and processing logic. This multi-functionality ensures consistent data handling while reducing maintenance burden, as there is only one path to maintain rather than two separate paths
3Adaptability or versatility
If internal refresh approach is used with separate paths for refresh and change data, then refresh functionality is provided, but productivity decreases due to inefficiencies in handling both types of data separately
Solution Approach 1:
By combining refresh data and change data into a single stream processed by the same capture service, the system eliminates redundant processing steps and improves data processing efficiency while maintaining full refresh functionality and adaptability
Data Source
AI summary
Data refresh in a replication environment is provided by: reading, by a data services component of a replication environment as part of a data refresh operation, a data store and identifying refresh data, from a source object of source objects of the data store, that is to be provided to a target system, where the data services component is further configured to read a replication log into which changes to source object(s) of the source objects are recorded, and send the changes as change data records to a capture service; and retrieving, by the data services component, the refresh data from the source object and sending the refresh data as refresh data records to the capture service, where the capture service is configured with data record handling routines for applying to received change data records and to the received refresh data records.