CXL Cache-Coherence Architecture With Adaptive Eviction Policies
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing demand for heterogeneous computing, driven by specialized workloads like compression, encryption, and artificial intelligence, necessitates high-performance connections between general-purpose processors and accelerators, requiring efficient memory and cache-coherence management to reduce overhead and latency.
Innovation Solution
The implementation of Compute Express Link (CXL) devices with cache buffers and switches, where cache eviction policies are dynamically adjusted based on cache hit ratios and latencies to optimize memory usage and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If multiple CXL devices are connected to share memory space to reduce overhead and latency, then memory access performance is improved, but cache coherence management complexity increases
Solution Approach 1:
The patent segments the cache coherence management by dividing CXL devices into different groups (first CXL devices and second CXL devices) with distinct eviction policies. This segmentation allows independent optimization of each group while maintaining overall system coherence, reducing the complexity of managing all devices uniformly.
Solution Approach 2:
The patent applies different cache eviction policies (first cache eviction policy for first CXL devices, second cache eviction policy for second CXL devices) tailored to the specific characteristics of each device group. This local quality approach optimizes performance for each segment while avoiding the need to manage all devices with a single complex policy.
2Productivity
If cache eviction policies are dynamically adjusted based on cache hit ratios, then cache efficiency is improved, but system control complexity increases
Solution Approach 1:
The patent implements feedback mechanisms where the host receives cache hit ratio information from CXL devices and uses this feedback to dynamically adjust eviction policies. The host compares cache hit ratios and reallocates policies accordingly, creating a closed-loop system that adapts to changing workload conditions.
Solution Approach 2:
The patent makes the cache eviction policies dynamic by allowing the host to modify which policy applies to which CXL devices based on real-time performance metrics. The system transitions from static, fixed policies to dynamic, adaptable policies that respond to actual cache hit ratios and system conditions.
3Productivity
If different cache eviction policies are allocated to different CXL devices, then performance optimization is improved, but policy management complexity increases
Solution Approach 1:
The patent segments CXL devices into distinct groups (first and second CXL devices) and assigns different eviction policies to each segment. This segmentation simplifies management by organizing devices into manageable categories with uniform policies within each group, rather than managing each device individually with unique policies.
Solution Approach 2:
The patent manages policy complexity by changing the allocation parameters (which policy applies to which device group) rather than managing individual device policies. The host adjusts high-level allocation parameters based on cache hit ratios, which automatically propagates to the affected devices, simplifying the management interface.
Data Source
AI summary
The present disclosure provides methods, apparatuses, and servers for cache-coherence. In some embodiments, an apparatus includes a plurality of compute express link (CXL) devices, and a switch. Each CXL device of the plurality of CXL devices includes a memory in which a portion of the memory is allocated as a cache buffer, to which different cache eviction policies are allocated. The different cache eviction policies are modified according to a cache hit ratio of the cache buffer. The switch is configured to connect the plurality of CXL devices to each other.