Data Replication via Log Entries and Consistency Points

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data replication methods are inefficient and resource-intensive, particularly for large databases, as they require significant processing power and storage space, and can burden the source system, leading to reduced performance and increased costs.

Innovation Solution

Implementing a system that monitors data operations, identifies a 'known good state' of an application, and replicates only the necessary data points, using log entries and consistency points to synchronize data between source and destination systems, thereby reducing the need for extensive data copying and minimizing the impact on the source system.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional data replication methods are used to copy large databases, then data backup and recovery capability is improved, but processing time and storage space requirements increase significantly

Engineering Contradiction:

Improvedata backup and recovery capabilityVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent extracts only the essential information needed for replication from the source database operations. Instead of copying entire data files, the system captures and transmits only the operational changes (log entries) that occur at consistency points, significantly reducing the volume of data that needs to be processed and transmitted while maintaining complete backup and recovery capability.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The replication process is segmented into discrete log entries representing individual operations at consistency points. Each log entry is an independent unit that can be processed, transmitted, and applied separately to the destination system. This segmentation allows for efficient processing of large databases by breaking down the replication task into manageable chunks rather than handling the entire database as a single unit.

Inventive Principle:

Principle #1Segmentation

2Reliability

If frequent data copying is performed to maintain data currentness, then data availability and accuracy are improved, but system performance and resource consumption deteriorate

Engineering Contradiction:

Improvedata availability and accuracyVSAvoidsystem performance

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system performs data replication periodically at consistency points rather than continuously copying all data changes. Consistency points serve as periodic checkpoints where the source database is in a known good state. By replicating only at these periodic intervals and transmitting only the log entries between points, the system maintains data currentness while minimizing the performance impact on the source system.

Inventive Principle:

Principle #19Periodic action

Solution Approach 2:

The patent applies partial action by transmitting only the necessary log entries required to bring the destination system up to the current consistency point, rather than copying all data or excessive amounts of redundant information. This selective transmission maintains data accuracy while reducing network bandwidth consumption and processing overhead.

Inventive Principle:

Principle #16Partial or excessive action

3Reliability

If entire large databases are copied to ensure complete data protection, then data recoverability is improved, but storage space requirements and processing overhead increase tremendously

Engineering Contradiction:

Improvedata recoverabilityVSAvoidstorage space

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The system extracts only the essential operational changes from the source database and stores these as log entries at the destination. Instead of maintaining a complete copy of the entire database, the destination system stores only the differential information (log entries) needed to reconstruct the database state at any consistency point, dramatically reducing storage space requirements while maintaining full recoverability.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent uses a selective copying approach where only log entries representing database operations at consistency points are copied to the destination system. This selective copying of essential information rather than complete data duplication enables the destination system to reconstruct the entire database state when needed, achieving complete data protection with minimal storage overhead.

Inventive Principle:

Principle #26Copying

Data Source

PatentUS8656218B2Memory configuration for data replication system including identification of a subsequent log entry by a destination computer

Publication Date: 2014.02.18 COMMVAULT SYSTEMS INC

AI summary

Performing data management operations on replicated data in a computer network. Log entries are generated for data management operations of an application executing on a source system. Consistency point entries are used to indicate a time of a known good, or recoverable, state of the application. A destination system is configured to process a copy of the log and consistency point entries to replicate data in a replication volume, the replicated data being a copy of the application data on the source system. When the replicated data represents a known good state of the application, as determined by the consistency point entries, the destination system(s) may perform a storage operation (e.g., snapshot, backup) to copy the replicated data and to logically associate the copied data with a time information (e.g., time stamp) indicative of the source system time when the application was in the known good state.