Log-Structure In-Place Data Modification for Write Contention

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing log-structure management systems face inefficiencies during write-intensive workloads due to contention from concurrent write operations and the accumulation of outdated entries, leading to increased processor and memory overhead.

Innovation Solution

In-place data modification within the log-structure segments, where the data field is updated while the time parameter is modified, allowing for reduced processor and memory utilization, and the prevention of outdated entry accumulation, along with the use of R-Pin and X-Pin mechanisms for concurrent access control.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If new and modified data are appended to the log head of the log-structure, then data integrity is maintained, but processor and memory overhead increase significantly during write-intensive workloads

Engineering Contradiction:

Improvedata integrityVSAvoidcomputational efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The log-structure is divided into multiple segments (log head segment, log body segments, log tail segment). By allowing in-place updates within log body segments while maintaining append-only behavior at the log head, the system segments write operations to reduce contention and improve parallelism, thereby resolving the contradiction between data integrity and computational efficiency

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system dynamically switches between two write modes: append-to-log-head for new entries and in-place-update for modifications. This dynamic approach allows the system to optimize for either data integrity (append mode) or computational efficiency (in-place update mode) depending on the operation type, resolving the contradiction

Inventive Principle:

Principle #15Dynamics

2Adaptability or versatility

If new and modified data are appended to the log head, then concurrent write operations can proceed, but contention for the single log head increases

Engineering Contradiction:

Improveconcurrent write supportVSAvoidwrite operation complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Inventive Principle:

Principle #15Dynamics

3Reliability

If bulk-copying full entries to the log head is performed, then data consistency is maintained, but processor and memory overhead become significant

Engineering Contradiction:

Improvedata consistencyVSAvoidprocessor and memory overhead

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent extracts only the modified portions of data (delta updates) rather than copying entire entries. When an entry is updated, only the changed fields are written to the log, significantly reducing the amount of data that needs to be processed and stored, thereby resolving the contradiction between data consistency and resource overhead

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The system changes the parameter of data transfer from full entry copying to selective field updating. By tracking which fields have changed and only logging those specific modifications, the system maintains data consistency while dramatically reducing processor and memory overhead during write operations

Inventive Principle:

Principle #35Parameter changes

4Quantity of substance

If outdated entries accumulate in the log-structure, then historical data is preserved, but storage space and processing efficiency decrease

Engineering Contradiction:

Improvehistorical data retentionVSAvoidprocessing efficiency

Core Design Contradiction:

Quantity of substanceVSProductivity

Solution Approach 1:

The system performs preliminary actions by maintaining indexes that map logical addresses to physical locations of data entries. This allows the system to quickly identify and manage outdated entries without scanning the entire log-structure, enabling efficient retention of historical data while maintaining processing efficiency through targeted operations

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11983159B2Systems and methods for management of a log-structure

Publication Date: 2024.05.14 HUAWEI TECH CO LTD

AI summary

An apparatus manages a data storage device storing data entries. The apparatus includes the data storage device configured to store a log-structure comprising segments each storing data entries, each data entry including a data field storing data and a time parameter indicative of an age of the data stored in the data field, and a processor configured to modify data stored in the data field of a certain data entry of a certain segment of the log-structure, and update the time parameter of the certain data entry according to the modified data. Modifying the data stored in the data field of the segment of the log-structure improves computational performance in comparison to, for example, other methods that append new and/or modified data to the single log head of the log-structure, which causes contention for the concurrent write operations.