Database Record Propagation via Change Consolidation and Thread Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for propagating data from staging to production databases in large, multi-channel environments are unreliable, prone to errors, and slow, especially in dynamic retail environments with frequent updates and 24/7 operations.
Innovation Solution
A customizable data propagation system that tracks and consolidates record changes, allowing selective propagation of approved changes using multiple threads, with customizable speed and robustness settings, and maintains a log of propagation status, enabling reliable and efficient synchronization between staging and production databases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If out-of-the-box data propagation products are used for large staging servers with many records and changes, then data propagation can be performed, but the system becomes unreliable, prone to errors, and impracticably slow
Solution Approach 1:
The patent divides the data propagation process into multiple independent threads, each handling specific tables or record sets. This segmentation allows parallel processing of data propagation tasks, significantly improving speed while maintaining reliability through controlled error handling in each thread.
Solution Approach 2:
The system dynamically adjusts propagation behavior based on real-time conditions, including pausing on errors, resuming after fixes, and adapting thread allocation. This dynamic approach optimizes both reliability and productivity by responding to system state changes rather than following fixed procedures.
2Reliability
If all record changes are propagated to ensure complete data synchronization, then production database is fully updated, but intermediary and duplicative changes increase errors and reduce efficiency
Solution Approach 1:
The system performs preliminary consolidation of record changes before propagation, identifying and removing intermediary or duplicative changes. This preliminary action ensures that only necessary changes are propagated, improving efficiency while maintaining complete synchronization through careful change selection.
Solution Approach 2:
The system tracks propagation status and provides feedback to identify and eliminate duplicative or intermediary changes. By monitoring what has been propagated and comparing it with current staging changes, the system avoids redundant operations and optimizes propagation efficiency.
3Speed
If data propagation is performed rapidly to meet 24/7 operational requirements, then response time is reduced, but error rate increases and system robustness decreases
Solution Approach 1:
The system dynamically adjusts propagation speed and behavior based on real-time error detection and system state. When errors are detected, the system can pause, correct, or resum propagation, maintaining both speed and reliability through adaptive control rather than fixed-speed processing.
Solution Approach 2:
The system incorporates error handling and rollback mechanisms that prepare for potential failures before they cause system-wide issues. This cushioning allows rapid propagation while maintaining robustness through pre-planned error recovery procedures.
Data Source
AI summary
A method and system for propagating database records from staging database to a production database so that the production database reflects record changes made in the staging database. The system of the present disclosure propagates record changes from the staging database to the production database in a flexible, customizable, and reliable way. The system consolidates database record changes to cull out any unnecessary record changes that do not need to be propagated. The system may also track the status of the database record changes, and send alerts related to the propagation status.