Parallel Truncation Point Management for Distributed Database Logs

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In distributed database systems, determining when to safely truncate a transaction log across multiple replication paths is challenging due to differences in data transmission and processing times, leading to potential data loss or inconsistency.

Innovation Solution

A method and system that receive and determine replicated truncation points from each replication path, using the oldest identifier to ensure all data has been replicated before truncating the transaction log, and employing communication interfaces and management systems to manage parallel replication paths.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If transaction log data is replicated to multiple replication systems via multiple parallel paths, then data redundancy and system reliability are improved, but it becomes difficult to determine when the transaction log data can be properly truncated

Engineering Contradiction:

Improvedata integrityVSAvoidtruncation management complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system implements feedback mechanisms where each replication system sends acknowledgments back to the primary system indicating successful replication status. The primary system collects these feedback signals from all replication paths and uses them to determine the safe truncation point, ensuring data integrity while managing multiple parallel paths systematically

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent introduces an intermediary coordination mechanism that manages the interaction between the primary database and multiple replication systems. This intermediary layer tracks replication status across all paths and coordinates truncation decisions, simplifying the complexity of managing multiple parallel replication paths

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If transaction log truncation is performed early to free up space, then productivity is improved, but data loss may occur if replication is not complete

Engineering Contradiction:

Improvetransaction log space utilizationVSAvoiddata safety

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system performs preliminary actions by pre-coordinating truncation decisions with all replication systems before actually truncating the transaction log. The primary system waits for confirmation from all replication paths that data has been successfully replicated before proceeding with truncation, ensuring data safety while maximizing space utilization

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system uses feedback from replication systems to determine the optimal truncation point. Each replication system provides feedback on its replication status, and the primary system uses this feedback to identify the safest point for truncation that maximizes space reuse while preventing data loss

Inventive Principle:

Principle #23Feedback

3Reliability

If waiting for all replication paths to complete before truncation, then data integrity is maintained, but time is lost due to slower space reuse

Engineering Contradiction:

Improvereplication consistencyVSAvoidtruncation delay

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system applies partial action by truncating the transaction log at the point where all necessary data has been replicated, rather than waiting for all possible replication operations to complete. This allows the system to reuse space for new transactions while maintaining replication consistency for the data that has been safely replicated

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The system performs preliminary coordination to identify the earliest safe truncation point before actually truncating. By pre-assessing the replication status across all paths and determining the optimal truncation point in advance, the system minimizes waiting time while ensuring data integrity

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS9607068B2Parallel truncation point management for log based replication

Publication Date: 2017.03.28 SYBASE INC

AI summary

Disclosed herein are system, method, and computer program product embodiments for replicating data in a distributed database system. Data containing a replicated truncation point associated with a replicating system is received via a data path. It can then be determined that the truncation point represents the point at which all data in a transaction log has been replicated (e.g., successfully or safely) and the transaction log can then be truncated at the truncation point (i.e., the data up to the truncation point deflected). Data containing an additional replicated truncation point associated with an additional replicating system via an additional data path may be received. It can then be determined that the additional replicated truncation point represents the point at which all data in the transaction log has been replicated and the transaction log can be then truncated at the additional replicated truncation point.