Timestamp-Based Data Block Comparison for Active-Active Databases
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for comparing large data tables in a mainframe Active-Active cluster solution are inefficient, taking hours to complete and can't identify data loss without disrupting production, and fail to recover from data loss.
Innovation Solution
A method involving selecting tables by timestamp, splitting them into data blocks, incrementally copying changes, and comparing data blocks between sites using SQL queries to ensure data consistency without interrupting the database management system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full count comparison is performed on large tables, then data consistency is verified, but comparison time becomes excessively long (several hours)
Solution Approach 1:
The patent divides large tables into data blocks based on partition ranges, allowing comparison of individual blocks rather than entire tables. This segmentation enables parallel processing and significantly reduces the time required for data consistency verification while maintaining comprehensive coverage through systematic block-by-block comparison.
Solution Approach 2:
The patent performs comparison on selected data blocks rather than complete tables, using sampling and selective block comparison based on change detection. This partial action approach verifies data consistency for critical blocks while avoiding unnecessary comparison of unchanged data, reducing overall comparison time while maintaining reliability.
2Reliability
If data comparison operation is performed on production tables, then data consistency is checked, but transaction rate is impacted
Solution Approach 1:
The patent performs data comparison operations on copied datasets rather than production tables. Change detection mechanisms identify modified data blocks, and comparisons are executed on these isolated copies. This preliminary action on replicas allows thorough consistency verification without blocking or degrading production transaction processing.
Solution Approach 2:
The patent introduces copied datasets as intermediaries between the comparison operation and production tables. These copies serve as buffers that absorb the comparison workload, allowing the system to verify data consistency on replicas while the production system continues operating at full transaction rate without interference.
3Reliability
If conventional comparison methods are used, then data comparison is performed, but data loss cannot be identified or recovered
Solution Approach 1:
The patent implements feedback mechanisms through change detection that track modifications to data blocks using timestamps and version information. When inconsistencies are detected during comparison, the system traces back through the feedback chain of change records to identify the first occurrence of data loss, enabling both detection and recovery by restoring from known good states.
Solution Approach 2:
The patent performs preliminary change detection and tracking before comparison operations, maintaining records of data block modifications with timestamps and version identifiers. This preliminary action creates a recovery roadmap that allows the system to quickly identify and restore from the first occurrence of data loss when inconsistencies are detected, rather than merely detecting the problem without recovery capability.
Data Source
AI summary
Tables are selected based on timestamp columns to obtain tables changed during a given time period and the selected tables are split into data blocks based on given partition ranges. Changes to data of the data blocks are incrementally copied into datasets. One or more tablespaces identified using a first structured query language query corresponding to a specified timestamp are unloaded based on output of the incrementally copying changes operation. Data of a source site and a target site are compared based on data blocks having the specified timestamp.