Sequence Engine Resequencing Data for Database Replication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data replication methods are cumbersome and inefficient, particularly in real-time processes, as they handle data in large segments and do not ensure accurate chronological order, which can compromise database integrity.
Innovation Solution
A sequence engine that captures data in real-time, resequences it based on chronological order, and applies it to a destination database, using a coupling facility and message broker to ensure accurate replication between databases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is handled in large segments in existing replication methods, then processing speed is improved, but chronological order accuracy deteriorates
Solution Approach 1:
The patent segments data into individual messages while maintaining sequence information, allowing parallel processing of multiple messages without losing chronological order. Each message is treated as an independent unit with embedded sequence data, enabling both high throughput and accurate ordering.
Solution Approach 2:
The patent introduces a sequence engine as an intermediary component between data capture and destination database. This mediator receives out-of-sequence messages, reorders them based on sequence information, and forwards them in correct chronological order, resolving the contradiction between speed and accuracy.
2Loss of time
If data replication is performed in real-time, then data freshness is improved, but system complexity deteriorates
Solution Approach 1:
The patent applies preliminary action by embedding sequence information in messages at the capture stage and pre-processing messages through the sequence engine before destination delivery. This upfront preparation enables real-time replication without complex runtime ordering logic at the destination.
Solution Approach 2:
Each message carries its own sequence information, making it self-identifiable and self-ordering. The messages are inherently equipped with the data needed for correct sequencing, reducing the complexity of external coordination mechanisms.
3Manufacturing precision
If sequence information is added to each message, then chronological ordering is improved, but message processing overhead deteriorates
Solution Approach 1:
The patent uses sequence information as a lightweight copy or identifier attached to each message, rather than duplicating entire message contents for ordering purposes. This minimal data addition enables precise ordering with negligible overhead.
Data Source
AI summary
According to one aspect, embodiments of the invention provide a sequence engine comprising a local queue configured to receive captured data in a first sequence, the captured data including a plurality of individual messages, each message including sequence information representative of a chronological order in which the messages was captured, a coupling facility in communication with the local queue, the coupling facility comprising a plurality of data tables wherein a first one of the plurality of data tables is configured to store the captured data in the first sequence, and a dispatcher in communication with the plurality of data tables and configured to resequence, based on the sequence information, the captured data into a second sequence, wherein the second sequence corresponds to the chronological order in which the plurality of individual messages were captured, and to forward the captured data to a destination database in the second sequence.