Data Caching Based on Popularity in Non-Uniform Memory Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed storage systems face issues with unreasonable data cache locations, leading to cache hotspots that affect access efficiency and database performance.
Innovation Solution
A method and apparatus for data processing that involve receiving access data from clients, determining the data popularity of target data based on this access data, and caching or migrating the data to a location within a non-uniform memory access structure that matches its popularity, thereby optimizing cache efficiency and avoiding hotspots.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is cached in a distributed storage system, then access efficiency is improved, but cache hotspots emerge leading to performance degradation
Solution Approach 1:
The patent applies local quality by differentiating cache locations into multiple levels (e.g., L1, L2 cache or different storage tiers) with varying access speeds and capacities. Different data types are selectively cached in different locations based on their access patterns, allowing high-access data to be placed in faster locations while preventing uniform distribution that causes hotspots.
Solution Approach 2:
The patent implements dynamic cache management by continuously monitoring data access patterns and dynamically adjusting cache locations based on real-time popularity metrics. The system can migrate data between different cache locations or even between nodes based on changing access patterns, preventing static cache configurations from creating persistent hotspots.
2Device complexity
If data popularity is not considered in cache location, then cache management is simplified, but cache efficiency decreases
Solution Approach 1:
The patent implements self-service through automatic cache management mechanisms that monitor data access patterns and autonomously adjust cache locations without requiring manual intervention. The system automatically identifies popular data and relocates it to optimal cache positions, balancing simplicity of operation with high cache efficiency.
Solution Approach 2:
The patent employs feedback mechanisms where access patterns are continuously monitored and fed back into the cache management system. This feedback loop enables the system to learn from actual usage patterns and adapt cache locations accordingly, achieving high cache efficiency while maintaining manageable complexity through automated decision-making.
3Device complexity
If data is stored in fixed locations, then system simplicity is maintained, but access performance is suboptimal
Solution Approach 1:
The patent transitions from static fixed locations to dynamic cache placement by continuously adjusting data storage positions based on access patterns. This dynamic approach allows the system to optimize access speed for frequently accessed data while maintaining overall system simplicity through automated management.
Solution Approach 2:
The patent applies parameter changes by modifying cache location parameters (such as cache level, node position, or storage tier) based on data popularity metrics. This allows the system to adapt storage parameters dynamically to match access patterns, improving access speed without requiring complex manual reconfiguration.
Data Source
AI summary
A method, apparatus, electronic device and storage medium for data caching based on data popularity is provided. In the method, first access data transmitted by at least one client is received. The first access data represents an instruction log of a remote direct data read instruction transmitted by the client for target data cached in a non-uniform memory access structure. A data popularity of the target data is obtained based on the first access data. The data popularity represents a frequency of the target data accessed by the remote direct data read instruction. Based on the data popularity of the target data, the target data is cached to a target location in a data storage unit implemented based on the non-uniform memory access structure. Alternatively, the target data out of the data storage unit is migrated. The target location has a data read-write speed corresponding to the data popularity.