Dual-Target Data Synchronization with View-Based Access Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data synchronization techniques for large electronic-commerce websites, particularly those using ORACLE™ databases, face issues such as undo/rollback table space exhaustion, long execution times, poor error tolerance, inefficiency, difficulty in resuming broken transports, and lack of monitoring capabilities, leading to performance degradation and system instability.
Innovation Solution
The implementation of a data synchronization system that establishes two target data tables alternately used for synchronization, utilizing an upper view to manage access and employing parallel processing and segment-wise commitment methods to reduce undo/rollback table space usage and improve efficiency, while also enabling error tolerance and monitoring of synchronization progress.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data synchronization is performed using traditional delete and insert operations on the same table, then data can be synchronized between backend data warehouse and frontend database, but undo/rollback table space is excessively consumed causing system performance degradation
Solution Approach 1:
The patent divides the synchronization process into distinct phases: first synchronizing data to a temporary table, then performing bulk insert operations to the target table, and finally deleting data from the temporary table. This segmentation separates the synchronization logic from the undo/rollback operations, reducing the space consumed in the undo/rollback table space while maintaining synchronization efficiency.
Solution Approach 2:
The patent creates a temporary table before performing synchronization operations. This temporary table serves as an intermediate storage area where data is first loaded from the backend data warehouse, and then transferred to the frontend database table. This preliminary action allows the system to perform bulk operations without continuously occupying undo/rollback table space.
2Reliability
If data synchronization operations are executed sequentially, then data can be synchronized accurately, but execution time increases causing long delays during synchronization
Solution Approach 1:
The patent performs preliminary data loading into a temporary table before the actual synchronization to the target table. This allows the system to prepare data in advance and then execute bulk insert operations, significantly reducing the time required for synchronization while maintaining accuracy through the structured two-step process.
Solution Approach 2:
The patent extracts the data loading operation into a separate temporary table, separating it from the main synchronization process. This extraction allows the system to load data efficiently into the temporary table first, then perform bulk operations to the target table, reducing overall execution time while maintaining synchronization integrity.
3Productivity
If traditional data synchronization methods are used, then data can be synchronized between databases, but error tolerance is poor and broken transports cannot be resumed
Solution Approach 1:
The patent implements feedback mechanisms through transaction logging and status tracking. When errors occur during synchronization, the system can identify the broken point through feedback information and resume from that specific location rather than restarting from the beginning. This feedback capability significantly improves error tolerance and allows broken transports to be resumed.
Solution Approach 2:
The patent performs preliminary error checking and validation before the main synchronization process. By validating data and checking for potential errors in advance, the system can identify and handle errors before they cause failures, improving overall error tolerance and enabling resumption of broken transports.
4Reliability
If data synchronization is performed during peak periods, then data can be kept up-to-date, but connection pool becomes fully occupied causing application server to stop functioning
Solution Approach 1:
The patent segments the synchronization process into background operations that run independently from the main application server. By using temporary tables and batch processing, the synchronization can be performed during peak periods without blocking application connections, thus maintaining both data freshness and application server functionality.
Solution Approach 2:
The patent introduces a temporary table as an intermediary between the backend data warehouse and the frontend database. This intermediary allows data to be loaded and processed in a separate space, reducing the impact on the main database connections and preventing the connection pool from becoming fully occupied during peak periods.
Data Source
AI summary
Data synchronization includes establishing a plurality of target data tables based on a source data table in which data to be synchronized is stored, determining a current target data table from the plurality of target data tables, synchronizing the source data table and the current target data table, and directing an application server to access the current target data table upon successful completion of synchronization.