Conditional Database Synchronization with Log-Based Update Gating
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cross-database synchronization methods, particularly in enterprise environments, are inefficient for large datasets due to sequential querying and updating of large batches of records, which are excessively time-consuming.
Innovation Solution
A method involving a processor that retrieves timestamps from a reference database, checks error and update indicators in a local log database, and conditionally updates a target database based on these indicators, using a pending flag to determine eligibility for updating, allowing parallel processing and reducing duplicate updates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If periodic sequential batch updates are performed to synchronize databases, then data consistency is maintained, but synchronization time becomes excessively long for large datasets
Solution Approach 1:
The patent segments the synchronization process into three distinct databases: a primary database storing source data, a secondary database storing target data, and a log database storing update status indicators. This segmentation allows parallel processing of multiple records simultaneously while maintaining data consistency through coordinated updates across all three databases.
Solution Approach 2:
The system performs preliminary actions by pre-calculating update indicators and error indicators in the log database before actual data synchronization occurs. These pre-computed indicators (update indicators showing whether target data needs updating, error indicators showing data quality issues) enable the system to quickly determine which records require synchronization without querying all records sequentially.
2Reliability
If sequential querying and updating of large batches of records is performed, then all records are synchronized, but the process becomes excessively time-consuming
Solution Approach 1:
The system implements self-service mechanisms where each record carries its own update indicator and error indicator in the log database. These self-contained metadata fields allow the synchronization system to autonomously determine which records need updating without requiring sequential queries to check each record's status, enabling parallel processing of multiple records simultaneously.
Solution Approach 2:
The log database serves as an intermediary between the primary and secondary databases, storing pre-computed update indicators and error indicators. This intermediary structure allows the synchronization system to efficiently query the log database for synchronization decisions rather than directly querying the primary database for each record, significantly improving throughput while maintaining completeness.
3Measurement precision
If multiple database queries are performed for each record update, then data accuracy is ensured, but network bandwidth is consumed and processing overhead increases
Solution Approach 1:
The system performs preliminary computation of update indicators and error indicators and stores them in the log database before synchronization occurs. This pre-computation eliminates the need for multiple runtime queries to determine whether updates are needed, reducing network bandwidth consumption while maintaining data accuracy through the use of pre-validated indicators.
Solution Approach 2:
The patent extracts critical synchronization decision information (update indicators and error indicators) from the primary database and stores them in the separate log database. This extraction allows the synchronization system to query only the compact log database for update decisions rather than performing multiple queries on the larger primary database, reducing network overhead while preserving data accuracy.
Data Source
AI summary
This disclosure describes techniques for conditionally updating a target database using reference data received from a reference database. In some cases, an update management system is configured to periodically retrieve reference timestamps from the reference database and determine whether corresponding fields in the target database should be updated based on those reference timestamps. The determination of whether to update the target database may be based on automated checks of error indicators and update indicators maintained in a log database that is local to the update management system. An error indicator may represent whether a data record is associated with any error conditions that would prevent updating the target database based on that data record. An update indicator may represent whether the reference timestamp associated with a data record has already been used to successfully update the target database.