Asynchronous Multi-System Consistency via Commit Log Event Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-system operations face delays and complexity due to the need for two-phase commit protocols, which are not suitable for loosely coupled systems and require operational constraint programming, making it difficult to ensure consistency without imposing synchronization delays or requiring complex constraint formulation.
Innovation Solution
The approach involves using commit logs to generate event streams that allow asynchronous execution of transactions across multiple systems, with correlation identifiers tying together changes made in different systems, enabling asynchronous eventual consistency without synchronization delays and without the need for operational constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If two-phase commit protocol is used to ensure consistency in multi-system operations, then system consistency is improved, but execution speed and productivity deteriorate due to synchronization delays
Solution Approach 1:
The system performs preliminary actions by making changes permanent in each database system immediately without waiting for other systems. The correlation identifier is injected into each database system's transaction in advance, enabling post-facto reconciliation without blocking execution. This resolves the contradiction by eliminating synchronization delays while maintaining consistency through eventual reconciliation.
Solution Approach 2:
The invention segments the multi-system operation into independent database transactions that can execute autonomously. Each system processes its changes separately with its own transaction ID, rather than requiring coordinated commitment. This segmentation allows parallel execution while the correlation identifier ties the segmented operations together for eventual consistency verification.
2Reliability
If two-phase commit protocol is used to ensure atomicity in multi-system operations, then operational reliability is improved, but device complexity increases due to coordination requirements
Solution Approach 1:
The invention extracts the coordination complexity from the transaction execution phase and relocates it to the post-facto reconciliation phase. The correlation identifier carries the necessary coordination information without requiring active coordination during execution. This removes the complex inter-system communication and waiting logic while preserving atomicity through eventual verification.
Solution Approach 2:
Each database system autonomously executes its transaction and makes changes permanent without external coordination. The systems self-manage their own transactions using their native transaction IDs, eliminating the need for complex two-phase commit coordination protocols. The correlation identifier enables self-verification through post-facto reconciliation.
3Reliability
If operational constraint conditions are programmed to prevent inconsistent states, then system consistency is improved, but ease of operation deteriorates due to programming complexity
Solution Approach 1:
The correlation identifier acts as an intermediary that automatically ties together changes across multiple database systems without requiring complex operational constraint programming. Instead of programming constraints to prevent inconsistency, the system uses the correlation identifier to automatically link transactions and verify consistency post-facto, greatly simplifying the programming model.
Data Source
AI summary
Techniques a provided for performing multi-system operations in which changes are asynchronously committed in multiple systems. Metadata about the multi-system operation is injected into the commit logs of one system involved in a multi-system operation. An event stream is generated based on the commit logs of the one system, and is used to drive the operations that one or more other systems need to perform as part of the multi-system operation. A reconciliation system reads the logs of all systems involved in the multi-system operation and determines whether the multi-system operation completed successfully. Techniques are also provided for using machine learning to generate models of normal execution of different types of operations, detect anomalies, pre-emptively send expectation messages, and automatically suggest and/or apply fixes.