CXL Cache Coherence with Adaptive Eviction Across Heterogeneous Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cache-coherence mechanisms in heterogeneous computing environments, such as those involving accelerators and general-purpose processors, face challenges in optimizing cache eviction policies to minimize latency and maximize performance due to varying device characteristics and workload demands.
Innovation Solution
A host device dynamically allocates and modifies cache eviction policies among CXL devices based on latency and cache hit ratios, using different policies like LRU and LFU to optimize cache usage and reduce latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single cache eviction policy is used for all CXL devices, then the system is simple to manage, but cache performance cannot be optimized for devices with different latency characteristics
Solution Approach 1:
The patent applies local quality by assigning different cache eviction policies to different CXL devices based on their specific latency characteristics. Instead of using a uniform policy across all devices, the system tailors the eviction strategy (e.g., LRU, LFU, or custom policies) to match each device's performance profile, thereby optimizing cache hit ratios for each individual device while maintaining overall system simplicity through automated host-side management
2Productivity
If cache eviction policies are dynamically modified based on performance metrics, then cache performance is optimized, but system complexity increases due to monitoring and modification mechanisms
Solution Approach 1:
The patent implements feedback by continuously monitoring cache performance metrics (such as hit ratios and latency) for each CXL device and using this information to dynamically adjust eviction policies. The host device collects performance data, compares it against thresholds or targets, and automatically modifies policies for devices that need optimization, creating a closed-loop system that adapts to changing workload conditions without requiring manual intervention
Solution Approach 2:
The system performs preliminary actions by pre-configuring multiple cache eviction policies and establishing performance thresholds before actual workload execution. When performance degradation is detected, the system can quickly switch to pre-prepared alternative policies, avoiding the need for complex real-time policy generation and reducing the computational overhead of dynamic optimization
3Productivity
If different cache eviction policies are assigned to CXL devices with different latency characteristics, then cache performance is optimized for each device, but the host must manage and coordinate multiple policies
Solution Approach 1:
The patent applies self-service by enabling the host device to automatically manage and coordinate multiple cache eviction policies without requiring manual configuration or intervention. The host monitors performance metrics, identifies devices that need policy changes, selects appropriate alternative policies, and applies modifications autonomously based on pre-defined criteria and thresholds, thereby simplifying host-side operations despite the complexity of managing heterogeneous devices
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure provides methods, apparatuses, and servers for cache-coherence. In some embodiments, an apparatus includes a plurality of compute express link (CXL) devices, and a switch. Each CXL device of the plurality of CXL devices includes a memory in which a portion of the memory is allocated as a cache buffer, to which different cache eviction policies are allocated. The different cache eviction policies are modified according to a cache hit ratio of the cache buffer. The switch is configured to connect the plurality of CXL devices to each other.