Network Storage Metadata Management via Core Manager
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current network storage systems face challenges with scalability, metadata server failure, load balancing, and disaster recovery due to limitations in metadata management, particularly in Storage Area Networks (SAN), where metadata synchronization mechanisms hinder performance and scalability.
Innovation Solution
Implementing a core manager in each storage server to centralize metadata in a common storage space, separating metadata from individual storage servers and managing it independently, allowing for dynamic scalability, failure handling, and load balancing without affecting data access performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If metadata servers use their own local storage space to store metadata, then each metadata server can achieve high data access performance and data storage efficiency, but the system cannot provide load balancing capabilities and disaster recovery capabilities
Solution Approach 1:
The patent divides metadata storage into two independent segments: hot metadata stored locally in each metadata server's memory for high-speed access, and cold metadata stored centrally in a shared storage system for reliability and backup. This segmentation allows each segment to serve its specific purpose optimally without compromising the other.
Solution Approach 2:
The patent introduces a shared storage system as an intermediary between metadata servers and the central metadata repository. This intermediary provides load balancing by distributing metadata access requests and ensures disaster recovery by maintaining centralized backups that can be restored if any individual server fails.
2Reliability
If metadata servers preserve synchronous copies of the overall metadata table, then load balancing and disaster recovery capabilities are improved, but the synchronization mechanism seriously damages performance
Solution Approach 1:
Instead of continuous synchronous replication, the patent implements periodic or event-driven metadata updates where changes are propagated to the shared storage system only when necessary. This periodic action reduces the overhead of synchronization while maintaining data consistency and load balancing capabilities.
Solution Approach 2:
The patent implements a dynamic metadata storage architecture where the distribution of metadata between local and shared storage can be adjusted based on access patterns and system load. This dynamic approach allows the system to optimize performance by keeping frequently accessed metadata local while maintaining centralized copies for reliability.
3Reliability
If each metadata server preserves a copy of the overall metadata table, then disaster recovery capability is improved, but the efficiency of metadata storage is reduced because each metadata record should have its corresponding local copy in every server
Solution Approach 1:
The patent applies local quality by storing different types of metadata in different locations based on their access characteristics. Hot metadata that requires frequent access is stored locally in each server's memory, while cold metadata is stored centrally in the shared storage system. This localized optimization reduces the complexity of maintaining copies across all servers while preserving disaster recovery capability.
4Quantity of substance
If storage servers are added to the SAN, then storage capacity is increased, but the overall I/O performance drops
Solution Approach 1:
The patent introduces a new dimension of metadata management by separating metadata storage from data storage. Metadata is managed independently in a shared storage system with its own optimization strategies, while data storage continues to expand in the traditional SAN architecture. This dimensional separation allows storage capacity to increase without degrading I/O performance, as metadata access patterns are optimized independently.
Data Source
AI summary
A data management method for network storage system that said network storage system includes a storage network, a cluster of storage servers that provide data storage services for application servers connecting to the storage network and storage space corresponding to each storage server, setting a core manager in storage server, said core manager centralizing the metadata of all storage servers in a common storage space; separating the metadata from said storage servers to make a storage server become a storage manager and the storage spaces corresponding to each storage server form the common storage space, allocating the storage space of metadata in said common storage space, and managing the corresponding relationship between metadata and said storage manager.