Distributed Storage Node Coordinate Mapping for Scalable Data Distribution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems without centralized management nodes, managing and processing object data items becomes inefficient as the number of storage nodes increases, leading to bottlenecks and difficulties in uniform distribution and biased storage based on data item properties.
Innovation Solution
A distributed storage system where data storage nodes are assigned node coordinates in a space identical to data coordinates, allowing for the selection of nodes with relatively short distances for storing object data items based on their identifiers, enabling uniform distribution and biased storage according to data item characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a large number of management nodes are used to manage storage nodes without centralized control, then scalability and distributed processing efficiency are improved, but it becomes difficult to record and manage storage nodes in a table due to synchronization complexity
Solution Approach 1:
The patent extracts the management function from the traditional centralized table-based approach and implements a distributed key-value store instead. Each management node maintains its own local key-value store, eliminating the need for complex table synchronization across multiple management nodes while maintaining the ability to record and manage storage node information.
Solution Approach 2:
The patent introduces a key-value store as an intermediary mechanism between management nodes and storage nodes. This intermediary allows management nodes to query and manage storage node information without requiring direct table synchronization, thus resolving the contradiction between distributed management and information consistency.
2Adaptability or versatility
If object data items are stored uniformly across all storage nodes, then system scalability is improved, but it becomes inefficient to process data items with specific properties that require biased storage in certain nodes
Solution Approach 1:
The patent applies local quality by allowing different storage strategies for different data items based on their properties. The key-value store enables biased storage where data items with specific properties can be directed to particular storage nodes or regions, while other data items are distributed uniformly, thus maintaining both scalability and processing efficiency.
Solution Approach 2:
The patent changes the storage parameter from uniform distribution to property-based biased distribution. By using the key-value store with customizable key structures, the system can dynamically adjust where data items are stored based on their properties, enabling efficient processing of specific data types while maintaining overall system scalability.
3Ease of operation
If fixed management nodes are used for centralized control, then table-based management of storage nodes is simplified, but bottlenecks form and scalability is limited
Solution Approach 1:
The patent segments the centralized management function into multiple distributed management nodes, each maintaining its own key-value store. This segmentation eliminates the bottleneck of fixed management nodes while maintaining management simplicity through the standardized key-value interface, thus resolving the contradiction between centralized control and scalability.
Data Source
AI summary
A distributed storage system comprises a plurality of data storage nodes that store a plurality of object data items in a distributed manner. The plurality of object data items are respectively characterized by a data coordinate and identified by a data identifier. The plurality of data storage nodes are respectively assigned a node coordinate in a space identical to that of the data coordinate. After selecting as first data storage nodes from among the plurality of data storage nodes a plurality of data storage nodes each assigned a node coordinate having a relatively short distance from a data coordinate characterizing each of the plurality of object data items, any of the object data item is respectively stored in a data storage node determined based on a data identifier that identifies the object data item from among the plurality of first data storage nodes.