Distributed Storage Node File Mapping via Ordered List
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed storage systems rely on centralized servers for configuration and reconfiguration, are complex to scale up or down, and require homogeneous hardware, leading to high administrative costs and limited flexibility.
Innovation Solution
A distributed storage system that uses a cluster of storage nodes with an ordered list to determine file storage locations, allowing nodes to independently manage file mappings and operate without a centralized server, enabling heterogeneous hardware and seamless scaling with minimal data redistribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a centralized server is used to monitor and configure distributed storage systems, then system administration and configuration can be simplified, but the system cannot be easily scaled up or down and has significant administrative costs
Solution Approach 1:
The patent removes the centralized server component from the distributed storage system, extracting the configuration and monitoring functions and distributing them across all storage nodes. Each node independently computes file mappings using a consistent hashing algorithm, eliminating the need for centralized configuration management while maintaining system scalability.
Solution Approach 2:
Storage nodes automatically perform configuration and reconfiguration tasks without centralized server intervention. When nodes are added or removed, the system self-adjusts through distributed computation of new mappings, with each node independently determining its role and file assignments based on the updated node list.
2Ease of operation
If homogeneous hardware is required for distributed storage systems, then administration and configuration are simplified, but the system lacks flexibility and cannot use heterogeneous hardware
Solution Approach 1:
The patent creates a universal software layer that runs identically on all storage nodes regardless of underlying hardware differences. The consistent hashing algorithm and mapping computation logic work the same way on heterogeneous hardware platforms, allowing the system to accept diverse hardware configurations while maintaining uniform administration through identical software behavior.
3Ease of operation
If centralized servers are used to propagate updates to storage servers, then configuration management is centralized, but reconfiguration complexity increases when scaling the system
Solution Approach 1:
Storage nodes autonomously compute their configuration and file mappings without receiving updates from centralized servers. When the node list changes, each node independently recalculates its mappings using the consistent hashing algorithm, automatically adapting to system changes without manual reconfiguration or centralized update propagation.
4Adaptability or versatility
If storage nodes are added or removed from the system, then system capacity can be adjusted, but significant data transfer is required to redistribute files
Solution Approach 1:
The patent uses consistent hashing to minimize data redistribution when nodes are added or removed. By changing only the hashing parameters (node list) rather than completely remapping files, the system ensures that most files remain on their original nodes, requiring data transfer only for files whose hash values now map to different nodes based on the updated configuration.
Data Source
AI summary
A system and method for distributing and accessing files in a distributed storage system uses an ordered list of the storage nodes in the system to determine the storage node on which a file is stored. The distributed storage system includes a cluster of storage nodes and may also include one or more client nodes that participate in the system as storage resources. Each node (client and storage) stores an ordered list of the storage nodes in the system, allowing any of the nodes to access the file. The list is updated whenever a new storage node is added to the system, an existing storage node is removed from the system, or a new storage node is swapped with an existing storage node. Each one of the nodes may independently compute a new mapping of files to the storage nodes when the ordered list is changed.