Applying Incremental Logs Across Distributed Storage Segments

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional database as a service (DBaaS) systems face limitations in data synchronization and redundancy, requiring each compute node to maintain a local replicated copy of the entire data, which can lead to storage capacity constraints and inefficient backup processes, especially in cloud computing environments.

Innovation Solution

A log service computing system distributes data across storage segment servers, logging incremental changes in a centralized persisted log, allowing secondary compute systems to perform write operations without maintaining full data replicas, and utilizing a storage service for quorum establishment and durability verification.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If each compute node maintains a local replicated copy of the entire data, then data availability and redundancy are improved, but storage capacity requirements and backup operation complexity increase

Engineering Contradiction:

Improvedata availabilityVSAvoidstorage capacity

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent extracts the log data from the compute nodes and stores it in a separate, centralized storage system. Only incremental log records are replicated to secondary compute nodes,而非 entire data sets. This extraction principle reduces storage requirements while maintaining reliability through selective replication of only the essential change data.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent segments data into log records that capture only incremental changes, rather than replicating complete data copies. The log is divided into discrete records that can be selectively applied, allowing secondary nodes to reconstruct data state through incremental application of log segments rather than maintaining full replicas.

Inventive Principle:

Principle #1Segmentation

2Reliability

If full data replication is performed for secondary compute nodes, then data consistency is improved, but backup operation time and setup duration increase

Engineering Contradiction:

Improvedata consistencyVSAvoidsetup duration

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by maintaining a persistent log of all data changes at the primary compute node before failover is needed. This pre-captured log data can be quickly applied to secondary nodes during setup or failover, eliminating the need for time-consuming full data replication while ensuring consistency.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Instead of copying entire data sets, the patent copies only log records that represent incremental changes. These compact log copies can be rapidly transmitted and applied to secondary compute nodes, dramatically reducing setup time while maintaining data consistency through selective replication of change data.

Inventive Principle:

Principle #26Copying

3Quantity of substance

If incremental log records are distributed to storage segment servers, then storage efficiency is improved, but system complexity for log management increases

Engineering Contradiction:

Improvestorage efficiencyVSAvoidlog management complexity

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent introduces a log service as an intermediary component that manages the complexity of log distribution and application. This mediator handles the coordination between primary compute nodes, storage segment servers, and secondary compute nodes, abstracting away the complexity of incremental log management while enabling storage efficiency through selective log record distribution.

Inventive Principle:

Principle #24Intermediary (Mediator)

4Quantity of substance

If secondary compute nodes apply log records from a centralized log service, then storage requirements are reduced, but network communication overhead increases

Engineering Contradiction:

Improvestorage requirementsVSAvoidnetwork communication overhead

Core Design Contradiction:

Quantity of substanceVSUse of energy by moving object

Solution Approach 1:

The system applies partial action by transmitting and applying only the necessary log records to secondary compute nodes, rather than full data sets. This partial replication of only incremental changes reduces storage requirements and optimizes network usage by transmitting minimal necessary data for maintaining consistency.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentEP3853734B1Applying a log to storage segments

Publication Date: 2025.09.10 MICROSOFT TECHNOLOGY LICENSING LLC

Figure 1
Figure 2~3
Figure 4

AI summary

Use of a log service computing system to apply a log to storage segments in the context of there being a primary compute system that operates upon data, and secondary compute system(s) that are at the ready to be promoted to the new primary compute system should the old primary compute system fail. Rather than have each of the primary and secondary compute systems maintain their own replicated of the entire data locally, the data is separated in storage segments that are collectively distributed amongst multiple storage segment servers.