Clustered Database Topology Change Control via Node Agent Streaming

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Clustered databases face challenges in maintaining topology changes, node consistency, and shard consistency, particularly during additions or removals of shards and nodes, where existing solutions struggle to ensure data integrity and synchronization across the database cluster.

Innovation Solution

A control computer monitors the clustered database, identifies topology changes, and configures node agents to extract and stream data images and operation logs to a secondary storage platform, versioning database versions with timestamps to maintain consistency across the cluster.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If topology changes are made to the clustered database (adding or removing shards or node computers), then the database can adapt to changing requirements and scale, but maintaining data integrity and synchronization across the cluster becomes more difficult

Engineering Contradiction:

Improvetopology change capabilityVSAvoiddata integrity

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The system performs preliminary actions by creating backup copies of data and operation logs before topology changes occur. The control computer monitors the clustered database and initiates backup processes in advance, ensuring that data integrity is preserved even when shards or nodes are added or removed. This preliminary backup action allows the system to maintain reliability during adaptive topology changes.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The control computer continuously monitors the clustered database for topology changes and provides feedback to adjust backup operations. When changes are detected, the system responds by synchronizing data and operation logs across the cluster, ensuring that data integrity is maintained throughout the topology transition. This feedback mechanism enables the system to adapt while preserving reliability.

Inventive Principle:

Principle #23Feedback

2Stability of the object's composition

If data and operation logs are synchronized across all nodes during topology changes, then node and shard consistency is maintained, but the complexity of managing the synchronization process increases

Engineering Contradiction:

Improvenode consistencyVSAvoidsynchronization management

Core Design Contradiction:

Stability of the object's compositionVSDevice complexity

Solution Approach 1:

The control computer acts as an intermediary that centralizes the management of synchronization operations. Instead of having each node independently manage synchronization, the control computer coordinates the extraction, backup, and distribution of data and operation logs across nodes. This intermediary approach maintains node consistency while reducing the complexity burden on individual nodes.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system creates copies of data and operation logs that can be distributed and synchronized across nodes without requiring complex real-time coordination. By using backup copies and versioning, the system simplifies the synchronization process while maintaining consistency, as nodes can replicate from established copies rather than continuously negotiating state.

Inventive Principle:

Principle #26Copying

3Reliability

If versioning with timestamps is implemented to track database changes, then data integrity and restoration capability are improved, but the storage requirements and processing overhead increase

Engineering Contradiction:

Improverestoration capabilityVSAvoidstorage requirements

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The system implements nested versioning where operation logs are organized in a hierarchical structure with timestamps. Rather than storing complete independent backups at each timestamp, the system nests incremental operation logs within version structures, allowing restoration to any point in time by applying logs sequentially. This nesting approach improves restoration capability while reducing overall storage requirements compared to full snapshots at each version.

Inventive Principle:

Principle #7Nested doll (Nesting)

Solution Approach 2:

The system applies partial versioning by maintaining operation logs and timestamps for all data changes, which provides complete restoration capability. The operation logs capture only the necessary change information rather than duplicating entire data sets, achieving full restoration capability with minimal additional storage overhead.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS11016952B2Systems and methods to process a topology change in a clustered database

Publication Date: 2021.05.25 RUBRIK INC

AI summary

Systems and methods to process a topology change in a clustered database are described. The system communicates a query to a source storage platform enquiring of a topology change in a clustered database stored on the source storage platform. The system receives a message, at a control computer responsive to communicating the query. The message includes node identifiers identifying nodes including a first node identifier identifying a first node included in the clustered database. The system automatically identifies an addition of the first node to the clustered database on the source storage platform. The system initializes the first node on the source storage platform by communicating a first node agent to the first node. The first node agent is configured to execute on the first node to extract the data image at the source storage platform and stream the data image to the secondary storage platform.