Distributed Storage Node Mapping via Ordered Lists

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing distributed storage systems face challenges in scalability and administrative complexity due to reliance on centralized servers and homogeneous hardware, making it difficult to add, remove, or reconfigure storage nodes without significant operational disruptions and high administrative costs.

Innovation Solution

A distributed storage system that uses a cluster of storage nodes and client nodes with an ordered list to determine file storage locations, allowing nodes to independently manage changes and operate without a centralized server, enabling heterogeneous hardware and minimal data redistribution during reconfiguration.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If a centralized server is used to monitor and configure storage nodes, then system administration is simplified, but the system cannot be easily scaled and administrative costs increase

Engineering Contradiction:

Improvesystem administrationVSAvoidscalability

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The patent removes the centralized server component from the system architecture. Each storage node independently determines file locations using a distributed hashing algorithm based on filename and ordered list of storage nodes, eliminating the need for centralized configuration and monitoring services.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Storage nodes autonomously compute file locations and handle configuration changes without external intervention. When storage nodes are added or removed, the system automatically redistributes files based on updated ordered lists, with each node independently calculating its new file assignments.

Inventive Principle:

Principle #25Self-service

2Device complexity

If homogeneous hardware is required for all storage nodes, then system configuration is simplified, but hardware flexibility and scalability are reduced

Engineering Contradiction:

Improvehardware configurationVSAvoidhardware flexibility

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The system accepts storage nodes with heterogeneous hardware configurations (different processors, memory, storage types). The software layer provides a universal interface that abstracts hardware differences, allowing any node to participate in the distributed storage system regardless of its specific hardware composition.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system dynamically adjusts file distribution parameters based on each storage node's actual capacity and performance characteristics. Storage capacity, I/O speed, and other parameters are measured and used to optimize file placement, allowing heterogeneous hardware to be efficiently utilized.

Inventive Principle:

Principle #35Parameter changes

3Adaptability or versatility

If storage nodes are added or removed from the system, then system capacity is adjusted, but significant data transfer and reconfiguration is required

Engineering Contradiction:

Improvesystem scalabilityVSAvoiddata transfer volume

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

Files are pre-distributed across storage nodes based on their expected capacity and position in the ordered list. When nodes are added or removed, the system calculates minimal redistribution paths, moving only the necessary files to new locations rather than reconfiguring the entire dataset.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The file distribution mapping is dynamically updated when storage nodes are added or removed. The system maintains an ordered list of storage nodes and uses this list to compute new file locations, enabling flexible scaling with minimal disruption to existing data.

Inventive Principle:

Principle #15Dynamics

4Adaptability or versatility

If any node can access any file, then system flexibility is improved, but determining file location becomes more complex

Engineering Contradiction:

Improveaccess flexibilityVSAvoidfile location determination

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent replaces complex centralized lookup tables or directory services with a mathematical hashing algorithm. Any node can determine the location of any file by computing a hash function using the filename and the ordered list of storage nodes, providing O(1) location determination without centralized coordination.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS8290919B1System and method for distributing and accessing files in a distributed storage system

Publication Date: 2012.10.16 ADEIA MEDIA HOLDINGS INC

AI summary

A system and method for distributing and accessing files in a distributed storage system uses an ordered list of the storage nodes in the system to determine the storage node on which a file is stored. The distributed storage system includes a cluster of storage nodes and may also include one or more client nodes that participate in the system as storage resources. Each node (client and storage) stores an ordered list of the storage nodes in the system, allowing any of the nodes to access the file. The list is updated whenever a new storage node is added to the system, an existing storage node is removed from the system, or a new storage node is swapped with an existing storage node. Each one of the nodes may independently compute a new mapping of files to the storage nodes when the ordered list is changed.