Edge Caching Deduplication via Reliability-Aware Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional edge caching methods face low cache hit rates due to the use of unreliable edge resources, which are not effectively addressed by existing deduplication and redundancy strategies, leading to inefficiencies in space utilization and file availability.
Innovation Solution
The proposed method, MEAN, employs a similarity-based hierarchical clustering (SHC) and heuristic algorithms to select file clusters for caching on edge servers, balancing deduplication and replication to optimize cache hit rates by considering file popularity, similarity, and server reliability, thereby improving space efficiency and availability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If traditional deduplication caching methods are used on unreliable edge servers, then space efficiency is improved, but file availability deteriorates
Solution Approach 1:
The patent applies partial replication by determining a replication factor for each file based on its popularity and the reliability of edge servers. Highly popular files are replicated more times to ensure availability, while less popular files use deduplication only. This partial action approach optimizes the balance between space efficiency and file availability by not uniformly replicating all files.
Solution Approach 2:
The patent dynamically adjusts the replication factor parameter based on file popularity metrics and server reliability assessments. By changing this parameter adaptively, the system transitions between deduplication and replication strategies for different files, resolving the contradiction between space efficiency and availability.
2Reliability
If more files are cached on edge servers to improve cache hit rate, then file availability is improved, but space efficiency deteriorates
Solution Approach 1:
Instead of caching complete replicas of all files, the system caches only the necessary portions based on popularity. Popular files are fully cached with appropriate replication, while less popular files use deduplication to share storage space. This partial caching strategy improves cache hit rates for popular files without proportionally increasing total storage consumption.
Solution Approach 2:
The patent uses selective copying where only popular files are replicated and cached on edge servers. Less popular files are stored once in a centralized location with deduplicated chunks. This copying strategy ensures that cached copies are available for frequent access while avoiding unnecessary duplication of less popular content.
3Loss of substance
If deduplication is applied to all files, then space efficiency is improved, but cache hit rate deteriorates due to unreliable servers
Solution Approach 1:
The patent applies different quality levels of storage protection to different files based on their popularity. Popular files receive higher quality protection through replication, ensuring they remain available despite server unreliability. Less popular files use standard deduplication. This local differentiation resolves the contradiction by providing enhanced reliability only where it matters most for cache performance.
Solution Approach 2:
The system applies partial replication only to popular files rather than all files. This selective approach provides the reliability needed to maintain cache hit rates for frequently accessed content while avoiding the space overhead of replicating less popular files, thus resolving the contradiction between space efficiency and cache performance.
Data Source
AI summary
A method and system for deduplication caching using an unreliable edge resource include acquiring a total storage capacity of all edge servers, searching for candidate cache files by a similarity-based hierarchical clustering (SHC) method, and acquiring file clusters of all the candidate cache files after clustering, where the candidate cache files each include a deduplicated data chunk, and based on the file clusters and an reliability of all of the edge servers, selecting, by a heuristic algorithm, a file cluster from the file clusters to cache to the edge server until a size of cached content reaches the total storage capacity. The present disclosure makes a trade-off between file availability and space efficiency, thereby effectively improving the cache hit rate in the limited edge caching space.