Polling Process for Missing Transaction Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In Polyglot Persistent systems, achieving consistent data representation across multiple databases is challenging due to the difficulty in maintaining Atomicity, Consistency, Isolation, Durability (ACID) properties, leading to the adoption of Eventual Consistency models, which can result in data loss and unacceptable lag times in polling methods and dependency on proprietary transaction log formats.
Innovation Solution
A novel polling process that identifies missing transaction identifiers by tracking contagious lists, allowing for efficient synchronization without introducing lag for quickly committed transactions, and reconciling uncommitted transactions by timestamping and removing invalid entries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a long lag time is introduced to allow committed data to synchronize, then data consistency is improved, but synchronization speed deteriorates
Solution Approach 1:
The system performs preliminary tracking of transaction identifiers as they are assigned, maintaining a contagious list that anticipates which transactions should have been committed by the next poll. This preliminary action allows the system to identify missing transactions immediately rather than waiting for a long lag period, thus maintaining data consistency without introducing unacceptable synchronization delays.
2Productivity
If polling frequency is increased to reduce lag time, then synchronization speed is improved, but system resource consumption worsens
Solution Approach 1:
The system uses the database's own transaction identifier assignment mechanism to track committed transactions. By leveraging the existing contagious sequence of transaction IDs that the database already generates, the polling process can efficiently identify which transactions should have been committed without requiring intensive external tracking or high-frequency polling, thus reducing processing power consumption while maintaining synchronization speed.
3Reliability
If transaction log parsing is implemented to ensure no changes are lost, then data completeness is improved, but system complexity worsens due to proprietary log formats
Solution Approach 1:
The invention extracts only the essential transaction identifier information from the database system rather than attempting to parse and interpret proprietary transaction log formats. By taking out just the transaction ID sequence that the database already provides in a standardized form, the system ensures data completeness without requiring complex, version-dependent log parsing software.
Data Source
AI summary
A poll of a first database is initiated (e.g., a SOT database). The poll of the first database is to identify if any transactions have occurred since a previous poll of the first database. Based on the poll of the first database, information is received that identifies one or more transaction identifiers. A missing transaction identifier is identified based on a contagious list of transaction identifiers. In response to identifying the missing transaction identifier, the missing transaction identifier is stored in a list of missing transaction identifiers for use in a later poll of the first database. By tracking the missing transaction identifier, the process can identify transactions that take longer to commit without introducing lag for transactions that commit quickly.