Asynchronous Namespace Maintenance in Distributed Storage

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In traditional distributed replicated data storage systems, maintaining namespace information across access devices is inefficient due to serial updates and processing overhead, especially when handling large numbers of data items, which increases traffic and processing load.

Innovation Solution

The method of asynchronous namespace maintenance, where batches of updates are stored and distributed using the object storage mechanism, reducing processing overhead by updating namespaces independently of actual file storage and leveraging system-defined batch sizes and time thresholds.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If serial updates are used to maintain namespace information across access devices, then namespace consistency is maintained, but processing overhead and network traffic increase significantly

Engineering Contradiction:

Improvenamespace consistencyVSAvoidprocessing overhead

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent segments namespace updates into batches grouped by access device. Instead of processing every update immediately and serially across all devices, the system accumulates updates into batches and processes them in parallel groups, reducing the processing overhead at each individual access device while maintaining overall namespace consistency.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements preliminary action by accumulating namespace updates into batches before distributing them to access devices. Updates are accumulated locally and then distributed in batches, allowing the system to prepare and organize update information in advance, reducing real-time processing overhead at access devices.

Inventive Principle:

Principle #10Preliminary action

2Loss of information

If real-time namespace updates are performed for every data item, then namespace information remains current, but network traffic and processing load increase

Engineering Contradiction:

Improvenamespace timelinessVSAvoidnetwork traffic

Core Design Contradiction:

Loss of informationVSLoss of energy

Solution Approach 1:

The patent implements periodic action by distributing namespace updates in batches at scheduled intervals rather than continuously. The system accumulates updates for a specified time period and then distributes them in periodic batches, reducing network traffic during transmission while ensuring namespace information remains current through timely batch updates.

Inventive Principle:

Principle #19Periodic action

Solution Approach 2:

The patent merges multiple individual namespace updates into single batch updates that are distributed together. By combining multiple updates into batches and distributing them as unified units, the system reduces the total number of network transmissions and processing operations required, thereby reducing network traffic and processing load while maintaining information timeliness.

Inventive Principle:

Principle #5Merging (Combining)

3Productivity

If asynchronous batch processing is implemented, then processing overhead is reduced, but namespace update timeliness may be compromised

Engineering Contradiction:

Improveprocessing efficiencyVSAvoidnamespace update delay

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent implements dynamics by making the batch processing parameters adjustable and adaptive. The system can dynamically configure batch size, time thresholds, and distribution timing based on actual workload conditions, allowing optimization of the balance between processing efficiency and update timeliness for different operational scenarios.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent applies parameter changes by allowing configuration of batch processing parameters such as time thresholds and batch sizes. The system can adjust these parameters to control the trade-off between processing efficiency and update timeliness, ensuring that namespace information is updated promptly enough for the application's needs while maintaining reduced processing overhead.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS9792344B2Asynchronous namespace maintenance

Publication Date: 2017.10.17 DATADIRECT NETWORKS INC

AI summary

Asynchronous namespace maintenance in a distributed replicated data storage system is disclosed. An access device/program serving as a front end to the distributed replicated data storage system updates a batch of updated meta data about stored data items when data items are stored in the distributed replicated data storage system. When the elapsed time since the last batch of data item meta data was stored exceeds a first threshold value or the current batch size exceeds a second threshold value, the access device/program stores the current batch of updated meta data as an object in the distributed replicated data storage system, receiving a batch object identifier for the stored batch of updated meta data, and distributes the batch object identifier to other access devices and/or access programs which retrieve the batch of updated meta data and update their namespaces.