Remote Index Node Lookup Using Hot-Spot Range Index Caching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed storage systems, accessing target data in storage nodes involves significant network transmission, leading to decreased data access performance due to recursive access of index structures.
Innovation Solution
Implement a popularity-aware range index caching solution that caches frequently accessed hot-spot nodes in the compute node, using a cache module to temporarily store index nodes and manage cache using heat statistics and management modules to reduce remote access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If recursive access of index structure is used to read target data, then data can be accurately retrieved, but network transmission increases and data access performance decreases
Solution Approach 1:
The patent pre-loads index nodes from the storage node into the compute node's cache before actual data access operations. This preliminary action stores frequently accessed index nodes (particularly hot-spot nodes at upper layers of the index tree) in local memory, so that when data access is needed, the compute node can directly access the cached index nodes without performing recursive network access to the storage node, thereby improving data access performance while maintaining retrieval accuracy
2Productivity
If all index nodes are cached in the compute node, then data access performance improves, but memory usage and cache management complexity increase
Solution Approach 1:
The patent implements differentiated caching strategies for different types of index nodes. It identifies and prioritizes caching of hot-spot nodes (nodes with high access frequency) at upper layers of the index tree, while allowing lower-layer leaf nodes to be accessed from storage when needed. This local quality approach optimizes cache usage by storing only the most frequently accessed index nodes locally, reducing cache management complexity while maintaining improved data access performance for the majority of operations
Data Source
Figure 1~2
Figure 3
Figure 4~5
AI summary
The present application discloses a method, apparatus, device for determining a remote index node, and a medium, belonging to the technical field of distributed storage. A range index cache is constructed for a compute node in a distributed architecture, and the cached index node is constructed in the range index cache. A target cached node covering a range of the determination request of the remote index node is determined by searching, and the remote index node in the storage nodes are determined according to the target cached node.