Metadata Caching for Storage Network Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing network-based storage systems experience unnecessary network latency and traffic due to the need for storage management commands to check data and structural integrity, which results in multiple requests being sent to storage servers, even if they violate these integrity rules.
Innovation Solution
Implementing a method and system for caching metadata at both the storage management client application and the network storage management server, allowing local checks of command validity and automatic updates of cached metadata, thereby avoiding unnecessary requests to the storage server.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If storage management commands check data and structural integrity by sending requests to storage servers, then command validity can be verified, but network latency and traffic increase due to multiple requests being sent
Solution Approach 1:
The patent applies preliminary action by maintaining and updating metadata caches at client applications before commands are sent to storage servers. The cached metadata contains current storage structure information, allowing clients to pre-validate commands locally against the cache before network transmission, thus avoiding unnecessary network round-trips for validity checking while ensuring command correctness
Solution Approach 2:
The patent implements copying by creating and maintaining local copies of metadata at client applications. These metadata copies are kept synchronized with the storage server through automatic update mechanisms, enabling clients to perform local validation operations without continuously querying the server, thereby reducing network traffic and latency while maintaining validation reliability
2Reliability
If storage management commands check data and structural integrity by sending requests to storage servers, then command validity can be verified, but network traffic increases due to multiple requests
Solution Approach 1:
The system performs preliminary validation of storage management commands against cached metadata before commands are transmitted over the network. This pre-checking mechanism identifies and blocks invalid commands locally, preventing unnecessary network traffic from carrying commands that would be rejected anyway, thus reducing overall network energy consumption
Solution Approach 2:
By maintaining local copies of metadata at client applications, the system enables distributed validation without centralized server involvement for every command. This copying approach eliminates redundant network communications between clients and servers for routine validation operations, significantly reducing network traffic volume and associated energy expenditure
3Loss of time
If metadata is cached locally at client applications, then network latency is reduced by avoiding unnecessary requests, but system complexity increases due to metadata synchronization requirements
Solution Approach 1:
The metadata caching system implements self-service through automatic update mechanisms. When the storage server detects metadata changes, it automatically pushes updates to client applications that have cached metadata. This self-updating approach eliminates the need for clients to continuously poll the server or implement complex manual synchronization logic, reducing system complexity while maintaining low latency
Solution Approach 2:
The system employs feedback mechanisms where the storage server monitors its own metadata state and automatically triggers update notifications to clients when changes occur. This feedback loop ensures metadata caches remain synchronized without requiring complex client-side synchronization protocols, simplifying the overall system architecture while achieving real-time consistency
Data Source
AI summary
A management server maintains a set of metadata describing a storage structure of a storage server. In response to a change of the storage structure, the management server automatically updates the set of metadata. The management server also manages information indicating what portion of the set of metadata is cached locally at a storage management client application. If that portion of the set of metadata is updated in response to the change of the storage structure, the management server sends information to update the metadata cached locally at the storage management client application so that the cached metadata is consistent with the set of metadata at the management server. By caching data at the storage management client application and at the network storage management server, the present invention advantageously avoids sending unnecessary requests to the storage server, thereby reducing network latency.