Database Segment Checksums for High-Availability Replication Integrity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High availability computer systems face challenges in maintaining data integrity during replication, particularly in relational databases, due to issues like data corruption, transmission errors, and inefficiencies in backup and synchronization methods, which can lead to discrepancies between primary and backup nodes.
Innovation Solution
A method is introduced to generate and store checksums for database segments on both primary and secondary nodes, using a journal to track modifications and verify data integrity, allowing for continuous monitoring and seamless integration with external replication systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If periodic complete backup is used to ensure data availability, then data can be restored in case of failure, but there is lag between backup operations causing data loss and significant network bandwidth consumption
Solution Approach 1:
The patent divides the database into segments and generates checksums for each segment individually. This allows incremental verification of only changed portions rather than complete periodic backups, reducing network bandwidth consumption while maintaining data availability through targeted synchronization of only modified segments.
Solution Approach 2:
The patent performs preliminary checksum generation and stores checksums in control tables before actual data replication occurs. This preliminary action enables efficient identification of changed segments, allowing the system to transfer only necessary data portions rather than performing complete periodic backups, thus reducing network bandwidth usage while ensuring data availability.
2Loss of energy
If incremental backup is used to reduce network bandwidth, then only changed data is copied, but a complete file is still transferred even when only a small portion was modified
Solution Approach 1:
The patent segments the database into manageable portions and generates individual checksums for each segment. This enables precise identification of only the specific segments that have changed, allowing transfer of minimal data portions rather than complete files, thereby improving backup efficiency while reducing network bandwidth consumption.
Solution Approach 2:
The patent replaces traditional mechanical file-based backup mechanisms with a checksum-based verification system. By using checksums stored in control tables to identify changed segments, the system substitutes inefficient complete file transfers with precise segment-level synchronization, significantly improving backup efficiency and reducing network bandwidth usage.
3Reliability
If data is continuously replicated to maintain high availability, then uptime is maximized, but data integrity verification becomes resource intensive and may cause downtime
Solution Approach 1:
The patent divides data verification into segment-level checksum comparisons rather than complete database verification. This segmentation allows integrity checking to be performed on small portions of data in parallel, maintaining high availability through continuous operation while reducing processing resource requirements and avoiding downtime associated with comprehensive verification.
Solution Approach 2:
The patent performs partial verification by checking only specific segments that may have changed, rather than verifying the entire database. This partial action approach maintains high availability by allowing continuous operation with minimal processing overhead, avoiding the resource-intensive complete verification that would cause downtime, while still ensuring data integrity through targeted checksum comparison.
Data Source
AI summary
A method for maintaining the integrity of a backup database table on a secondary node against a continuously replicated and dynamically changing original database table on a primary node includes generating an original checksum of a segment of the original database table between a beginning record and an ending record. The checksum and identification data may be stored in an original control database table. The original control database table may be replicated to a backup control database table. Thereafter, a backup checksum of a segment of the backup database table between a beginning record and an ending record is generated. A comparison between the backup checksum and the original checksum is made in response to a triggering update to the backup control database table. Collisions by application generate transactions in the source and target databases may be detected despite the use of third party “black box” replication processes.