Distributed Cache File Access via Decentralized Metadata Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed cache systems face performance bottlenecks and increased cache pressure due to centralized metadata management and network delays in both tree and centralized structures, necessitating a more efficient file access method.
Innovation Solution
Implement a distributed cache system where metadata information is maintained by first service nodes, with files divided into segments and cached across multiple second service nodes, allowing direct access to cached data without frequent metadata updates, and incorporating load balancing and backup nodes to manage node abnormalities and loads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a tree structure is used in distributed cache system, then file access can be organized hierarchically, but the root node and child node adjacent to root node become performance bottlenecks
Solution Approach 1:
The patent segments the centralized metadata management function by allowing each client node to independently maintain metadata for specific files it caches. This distributes the metadata management burden across multiple nodes rather than concentrating it at the root node, thereby eliminating the performance bottleneck while maintaining hierarchical organization through segmented metadata responsibility.
Solution Approach 2:
The patent extracts the metadata management function from the root node and assigns it to individual client nodes that cache the corresponding files. Each client node takes out the responsibility of maintaining its own metadata independently, removing the dependency on root node for metadata operations and thus resolving the bottleneck issue.
2Speed
If each node caches file data as much as possible in tree structure, then local access speed improves, but cache pressure on each node increases significantly
Solution Approach 1:
The patent applies local quality by having each client node maintain metadata only for the specific files it caches locally, rather than maintaining all file metadata. This localized metadata management reduces the memory burden on each node while enabling fast local access to frequently used files, as nodes only store and manage metadata for their own cached content.
3Device complexity
If centralized structure with metadata server is used, then metadata management is centralized, but load on metadata server increases and network requests increase
Solution Approach 1:
The patent implements self-service by enabling each client node to independently manage its own metadata without requiring constant communication with a central metadata server. Nodes autonomously maintain their metadata, perform local lookups, and manage their cached files, thereby eliminating the performance bottleneck and high network traffic associated with centralized metadata management.
4Reliability
If each node accesses metadata server for file access in centralized structure, then centralized control is maintained, but number of network requests increases and file reading delay increases
Solution Approach 1:
The patent applies preliminary action by having client nodes obtain and cache metadata locally when they first access a file, so that subsequent file access operations can proceed without additional network requests to the metadata server. This preliminary metadata caching eliminates repeated network round-trips and reduces file reading delay while maintaining system control.
Data Source
AI summary
A file access method for a distributed cache system, a medium and a device are provided. The method includes: reading metadata information of a target file from a first service node of the target file in response to an access request for the target file, wherein the first service node of the target file is any client node in the distributed cache system, a plurality of client nodes in the distributed cache system are respectively used as first service nodes of different files, and one first service node is used to maintain metadata information of at least one file; and accessing a second service node used to cache the target file according to the metadata information, and reading the target file from the second service node, wherein the second service node is any client node in the distributed cache system.