Adaptive Network Cache Redundancy Management for WAN Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deduplication systems in datacenters face latency issues due to cache miss penalties and synchronization problems, which increase bandwidth utilization and delay data transmission across wide area networks.
Innovation Solution
A distributed network system with a local cache management system that includes an index engine and a redundancy management engine, which uses adaptive emission to select transmission algorithms based on expected latency costs and determines whether data is stored in a remote cache, optimizing data transmission by only sending necessary segments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If a deduplication cache is searched to determine whether data is stored within the cache, then data redundancy is reduced, but latency increases due to cache miss penalty
Solution Approach 1:
The system performs preliminary actions by proactively synchronizing cache fingerprints with remote caches before data transmission is needed. The local cache management system maintains an index of fingerprints and periodically updates remote caches, so that when data needs to be transmitted, the system can quickly determine availability without performing full cache searches, thereby reducing latency while maintaining deduplication benefits
Solution Approach 2:
The patent introduces an intermediary mechanism using fingerprint indexes and synchronization protocols between local and remote cache management systems. Instead of directly searching entire caches, the system uses fingerprint indexes as intermediaries to quickly determine data availability and coordinate transmissions, reducing the time cost of cache misses
2Loss of energy
If deduplication caches are used to store frequently used data, then bandwidth utilization is reduced, but cache synchronization issues arise between multiple caches
Solution Approach 1:
The system implements feedback mechanisms where the local cache management system continuously monitors and reports cache status to remote caches. When data is added or modified locally, the system sends feedback signals to update remote cache indexes, ensuring all caches remain synchronized without requiring constant full-cache synchronization, thus maintaining reliability while reducing bandwidth usage
Solution Approach 2:
The patent segments the cache synchronization problem by dividing the cache into manageable units and synchronizing them individually through fingerprint indexes. Instead of synchronizing entire caches, the system synchronizes only the fingerprint metadata segments, which reduces the complexity and improves reliability of the synchronization process across multiple distributed caches
3Adaptability or versatility
If data is transmitted through WAN between datacenters, then data sharing is enabled, but bandwidth constraints and transmission delays are introduced
Solution Approach 1:
The system performs preliminary caching and indexing of data fingerprints at multiple datacenters before actual data transmission is needed. By pre-synchronizing the fingerprint indexes and preparing data for potential transmission, the system reduces the impact of WAN latency when data sharing is actually required, enabling fast data sharing without being constrained by WAN bandwidth limits
Data Source
AI summary
Embodiments include a local cache management system that is configured to be coupled to a local cache and that includes an index engine configured to store fingerprints of message segments stored in the local cache and a redundancy management engine coupled to the index engine. The redundancy management engine includes an adaptive emitter configured to receive a message segment to be transmitted to a remote device, determine expected latency costs of a plurality of transmission algorithms, and select a transmission algorithm, such as by selecting the lowest expected latency cost. The adaptive emitter is also configured to determine whether the message segment is stored within a remote cache management system associated with the remote device, and transmit the message segment through a network to the remote cache management system using the selected transmission algorithm upon a determination that the message segment is not stored within the remote cache management system.