Immutable Key-Value Storage for Distributed Systems

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Distributed storage systems that implement quorum-based consensus algorithms face challenges such as complexity in building and testing, inability to perform read/write operations with an insufficient number of storage nodes, and difficulty in scaling due to reliance on a predefined number of replica storage nodes.

Innovation Solution

A distributed storage system that stores data as immutable key-value entries, allowing read/write operations without requiring a quorum, by creating new key-value entries for updated data and using a minimal set of storage nodes for replication, enabling read/write operations with less than a quorum of storage nodes and facilitating scalable key-value entry management.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If quorum-based consensus algorithms are implemented to ensure data consistency, then reliability is improved, but device complexity increases and ease of operation deteriorates

Engineering Contradiction:

Improvedata consistencyVSAvoidsystem complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent extracts the quorum consensus mechanism from the storage system and replaces it with a simpler key-value entry approach. Each storage node independently stores and retrieves key-value pairs without requiring coordination with other nodes for consistency, thereby eliminating the complex consensus algorithm while maintaining data accessibility and basic consistency through replication.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent segments the monolithic quorum consensus process into independent key-value storage operations at each node. Instead of requiring collective decision-making for every read/write operation, the system divides data into key-value pairs that can be independently stored, retrieved, and replicated across nodes without coordination overhead.

Inventive Principle:

Principle #1Segmentation

2Reliability

If quorum-based consensus algorithms are used to maintain data consistency, then reliability is improved, but ease of operation worsens due to inability to perform read/write with insufficient nodes

Engineering Contradiction:

Improvedata consistencyVSAvoidread/write accessibility

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The patent introduces dynamic adaptability by allowing the system to function with any number of available storage nodes. Unlike rigid quorum systems that fail when threshold nodes are unavailable, the key-value approach dynamically adapts to node availability, allowing read/write operations to proceed with whichever nodes are currently accessible, thereby improving operational flexibility and availability.

Inventive Principle:

Principle #15Dynamics

3Reliability

If quorum-based consensus algorithms operate with predefined replica nodes, then data consistency is maintained, but adaptability deteriorates due to difficulty in scaling

Engineering Contradiction:

Improvedata consistencyVSAvoidscalability

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The patent creates a universal storage interface where each node can independently store and retrieve key-value pairs without being constrained by predefined replica roles. This universal approach allows any node to perform any storage operation, enabling seamless scaling as nodes are added or removed from the system without reconfiguring consensus groups or replica assignments.

Inventive Principle:

Principle #6Universality (Multi-functionality)

4Reliability

If quorum-based consensus algorithms are implemented, then data consistency is improved, but building and testing effort increases significantly

Engineering Contradiction:

Improvedata consistencyVSAvoidbuilding and testing effort

Core Design Contradiction:

ReliabilityVSEase of manufacture

Solution Approach 1:

The patent adopts a simpler, more disposable approach to consistency by using independent key-value storage at each node rather than maintaining complex consistent state across all nodes. Each node independently manages its own key-value pairs, eliminating the need for expensive and complex consensus coordination, logging, and recovery mechanisms that would require extensive building and testing effort.

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

Data Source

PatentUS11256717B2Storage of key-value entries in a distributed storage system

Publication Date: 2022.02.22 VMWARE INC

AI summary

A distributed storage system, such as a distributed storage system in a virtualized computing environment, stores data in storage nodes as immutable key-value entries. A coordinator storage node creates a key-value entry and attempts to store the key-value entry in the coordinator storage node and in neighbor storage nodes. If the storage of the key-value entry in the in the coordinator storage node and in the neighbor storage node is successful, the coordinator storage node pushes the key-value entry to other storage nodes in the distributed storage system for storage as replicas.