Distributed Storage Management Microservices Architecture

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional data storage management systems face bottlenecks due to centralized control, leading to processing overload, slow database access, and network overload, which limit system performance and growth, especially in large-scale environments with tens of thousands of clients.

Innovation Solution

The implementation of a distributed and scalable storage management architecture using a client-based or microservices-based approach, where management responsibilities and data are distributed away from the central storage manager, allowing clients to perform storage operations autonomously with minimal communication to the central storage manager, and utilizing micro-servers to service client groups, reducing reliance on the central database.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If a centralized storage management system is used, then management control is simplified and centralized, but system performance deteriorates and bottlenecks occur when tens of thousands of clients are involved

Engineering Contradiction:

Improvecentralized management controlVSAvoidsystem performance

Core Design Contradiction:

Ease of operationVSProductivity

Solution Approach 1:

The patent segments the centralized storage management system into distributed storage nodes that operate autonomously. Each node manages its own storage operations locally without requiring constant communication with a central controller, thereby eliminating bottlenecks while maintaining management effectiveness through distributed coordination protocols.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a distributed file system layer as an intermediary between clients and physical storage nodes. This intermediary handles management operations locally at each node, reducing the need for direct communication with central controllers and improving overall system performance while maintaining centralized policy enforcement where needed.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Device complexity

If management data is centralized in a database at the central controller, then management is streamlined, but communication overhead and network overload increase with large numbers of clients

Engineering Contradiction:

Improvemanagement structureVSAvoidcommunication overhead

Core Design Contradiction:

Device complexityVSLoss of energy

Solution Approach 1:

The patent implements local quality by caching frequently accessed management data and metadata locally at each storage node rather than requiring all clients to access the central database. This local caching reduces network communication overhead significantly while maintaining data consistency through periodic synchronization with the central management system.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent performs preliminary action by pre-loading and caching management metadata, access permissions, and storage policy information at each distributed node before they are needed. This preliminary local availability of management data eliminates the need for real-time database access during storage operations, reducing communication overhead and network load.

Inventive Principle:

Principle #10Preliminary action

3Reliability

If clients communicate frequently with the central storage manager, then storage operations can be coordinated, but processing overload and slow database access occur

Engineering Contradiction:

Improvestorage operation coordinationVSAvoiddatabase access time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent implements dynamic operation modes where storage nodes can switch between centralized coordination mode (for policy changes and administrative operations) and autonomous operation mode (for routine storage operations). This dynamic adaptation allows the system to maintain coordination reliability when needed while minimizing database access time during high-volume storage operations by operating autonomously.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent uses periodic action by implementing batch synchronization protocols where storage nodes periodically update and synchronize their local state with the central management database, rather than communicating continuously. This periodic synchronization maintains data consistency and coordination reliability while dramatically reducing the frequency of database access operations and associated delays.

Inventive Principle:

Principle #19Periodic action

Data Source

PatentUS10893103B2Distributed and scalable storage management using a storage-microservices server

Publication Date: 2021.01.12 COMMVAULT SYSTEMS INC

AI summary

Embodiments are described for distributed and scalable client-based storage management and microservices-based storage management, which can operate in combination. Embodiments distribute responsibilities and client-specific information needed for storage management jobs away from a central storage manager and its management database. New features and components minimize communications to/from the storage manager and ensure the integrity and synchronization of data being distributed. The management database at the storage manager retains its role as repository of information for the data storage management system as a whole, yet without being a bottleneck to storage operations. Improvements are implemented within the storage manager, through new functionality added to clients, and further through a new micro-server component interposed between storage manager and clients.