Distributed Storage Metadata Segmentation for Heterogeneous Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage systems face challenges in scaling with improving storage device performance, particularly with latencies in the order of microseconds, and are inadequate in handling heterogeneous networks, leading to bottlenecks in systems software and storage protocol overhead, as well as complex and inflexible configurations.
Innovation Solution
A distributed storage system that organizes and maintains metadata in a network, allowing arbitrary association of metadata and data across nodes, enabling efficient handling of I/O operations, replication, and flexible device management, without relying on fixed locations, and supports heterogeneous devices like SSDs and HDDs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If storage systems use fixed-location metadata and organized device tiers, then device management and data access are simplified, but system flexibility and adaptability to heterogeneous devices deteriorate
Solution Approach 1:
The patent segments the storage system into distributed nodes, each independently managing its own metadata and data storage. This segmentation allows each node to operate autonomously with simple local metadata management while the collective system achieves high adaptability to heterogeneous devices through the distributed architecture. Each node can handle different device types without requiring global reconfiguration.
2Reliability
If storage systems are designed with fixed device organization and protocols, then system stability and reliability are improved, but performance scaling with faster devices deteriorates due to software overhead
Solution Approach 1:
Each distributed node in the patent serves itself by maintaining local metadata and making autonomous decisions about data access and replication. This self-service approach eliminates the need for centralized coordination and complex protocol overhead, allowing the system to scale with faster devices while maintaining reliability through distributed consensus mechanisms for critical operations.
3Reliability
If metadata is maintained centrally in fixed locations, then data consistency and reliability are improved, but system complexity and management overhead increase
Solution Approach 1:
The patent segments metadata management across multiple distributed nodes rather than maintaining it centrally. Each node maintains metadata for the data it stores locally, eliminating the single point of failure and reducing the complexity burden on any single component. Data consistency is maintained through replication and distributed consensus protocols.
Solution Approach 2:
Each distributed node in the patent maintains metadata locally rather than accessing it from a central location. This local quality approach reduces access latency and simplifies the operational complexity for each node, as they only need to manage their own local metadata while the distributed system as a whole maintains data consistency through replication.
4Ease of operation
If storage systems use centralized metadata management, then data access coordination is simplified, but system scalability and performance deteriorate due to coordination overhead
Solution Approach 1:
The patent segments data access coordination across distributed nodes, where each node independently coordinates access to its local data using locally-stored metadata. This eliminates the centralized coordination bottleneck and allows parallel access operations across the distributed system, significantly improving I/O performance while maintaining coordinated access through distributed protocols.
Data Source
AI summary
A storage system for organizing and maintaining metadata in a distributed network. The system includes: a network; a plurality of distributed nodes configured to communicate through the network; a plurality of block devices configured to communicate with the plurality of distributed nodes through the network; and a management server configured to communicate with each of the plurality of distributed nodes and block devices and further configured to arbitrarily associate metadata and data to any of the plurality of distributed nodes such that a global copy of the metadata does not exist in one or more fixed locations known to all distributed nodes.