Distributed Storage Management Architecture for Scalable Client Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional data storage management systems face bottlenecks due to centralized control, leading to processing overload, slow database access, and network overload, which limit system performance and growth, especially in large-scale environments with tens of thousands of clients.
Innovation Solution
The implementation of a distributed and scalable storage management architecture using a client-based or microservices-based approach, where management responsibilities and data are distributed away from the central storage manager, allowing clients to perform storage operations autonomously with minimal communication to the central system, and utilizing micro-servers to service client groups, reducing reliance on the central storage manager.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a centralized storage management system is used, then management control is simplified and centralized, but system performance degrades and bottlenecks occur when scaling to tens of thousands of clients
Solution Approach 1:
The patent segments the centralized storage management system into distributed storage nodes that operate autonomously. Each node manages its own storage operations independently, eliminating the single-point bottleneck of centralized control. The system divides management responsibilities across multiple nodes, allowing parallel processing and improved scalability while maintaining simplified operational interfaces through coordinated node interactions.
2Stability of the object's composition
If a centralized management database is used, then data consistency is maintained, but database access becomes slow and communication resources are insufficient at scale
Solution Approach 1:
The patent extracts the centralized management database into distributed data structures stored locally at each storage node. Each node maintains its own data consistency independently through local data structures, eliminating the need for frequent centralized database access. This extraction reduces communication overhead and database access time while maintaining data consistency through distributed coordination mechanisms.
3Stability of the object's composition
If centralized control is implemented, then storage operations are coordinated, but processing overload and network overload occur in large-scale systems
Solution Approach 1:
The patent implements dynamic distributed coordination where storage nodes autonomously coordinate operations based on local conditions and peer-to-peer communication. The system transitions from static centralized control to dynamic distributed decision-making, allowing nodes to adapt to changing system conditions independently. This reduces processing overload at any single point and distributes network traffic across multiple coordination channels.
Data Source
AI summary
Embodiments are described for distributed and scalable client-based storage management and microservices-based storage management, which can operate in combination. Embodiments distribute responsibilities and client-specific information needed for storage management jobs away from a central storage manager and its management database. New features and components minimize communications to/from the storage manager and ensure the integrity and synchronization of data being distributed. The management database at the storage manager retains its role as repository of information for the data storage management system as a whole, yet without being a bottleneck to storage operations. Improvements are implemented within the storage manager, through new functionality added to clients, and further through a new micro-server component interposed between storage manager and clients.