Distributed Storage Volume Allocation via Dynamic Segment Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face challenges in maintaining data consistency and efficiency due to their distributed nature, leading to delays and storage overhead in managing and retrieving large amounts of data across multiple storage nodes.
Innovation Solution
A storage scheme that allocates logical storage volumes across multiple storage nodes, using thin-provisioning to scale storage gradually, and employs a volume map, slice map, segment map, and block map to manage data distribution, reassign slices as needed, and maintain data consistency through efficient data movement and indexing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is distributed across multiple storage nodes, then storage capacity and scalability are improved, but data consistency and retrieval efficiency deteriorate due to coordination overhead
Solution Approach 1:
The patent segments data into chunks and distributes them across multiple storage nodes in a hierarchical structure. Each node maintains local metadata, and the system uses a tiered architecture where data is organized into pools, volumes, and files. This segmentation allows the system to scale storage capacity while maintaining consistency through distributed metadata management and coordinated access protocols.
2Adaptability or versatility
If data is distributed across multiple storage nodes, then storage scalability is improved, but system complexity increases due to coordination and management overhead
Solution Approach 1:
The patent introduces intermediary components including a metadata server, namespace manager, and storage pool manager that mediate between clients and distributed storage nodes. These intermediaries abstract the complexity of distributed data management, providing unified namespace management, consistent access protocols, and coordinated data operations while enabling the system to scale transparently.
Solution Approach 2:
The patent implements universal storage pools that can accommodate multiple types of storage media (HDD, SSD, tape) and support various data access patterns through a unified interface. The storage system provides multi-functional capabilities including hierarchical storage management, data replication, snapshots, and different access protocols, reducing the need for separate specialized systems.
3Ease of operation
If traditional storage allocation is used, then data access is simple, but storage efficiency and resource utilization deteriorate due to fixed allocation overhead
Solution Approach 1:
The patent implements dynamic storage allocation where storage capacity is not fixed but adapts based on actual data needs. The system uses thin provisioning to allocate storage space on-demand, allows automatic tiering that dynamically moves data between storage tiers based on access patterns, and provides flexible volume expansion without data migration. This dynamic approach maintains simple data access while dramatically improving storage efficiency and resource utilization.
Data Source
AI summary
A storage scheme allocates portions of a logical volume to storage nodes in excess of the capacity of the storage nodes. Slices of the storage nodes and segments of slices are allocated in response to write requests such that actual allocation on the storage nodes is only in response to usage. Segments are identified with virtual segment identifiers that are retained when segments are moved to a different storage node. Logical volumes may therefore be moved seamlessly to different storage nodes to ensure sufficient storage capacity. Data is written to new locations in segments having space and a block map tracks the last segment to which data for a given address is written. Garbage collection is performed to free segments that contain invalid data, i.e. data for addresses that have been subsequently written to.