Fabric Switch Smart Caching for FAM Latency and Congestion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Efficient use of fabric-attached memory (FAM) resources is challenging due to arbitrary data storage, leading to overloaded FAM nodes and higher memory access latencies, which degrade application performance and lower bandwidth utilization.
Innovation Solution
Implement a cache agent within network switches to manage caching based on data access parameters, such as latency and sparsity ratio, dynamically reallocating cache resources to balance load across FAM components and reduce congestion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored arbitrarily in fabric-attached memory, then memory capacity is maximized, but memory access latency increases and bandwidth utilization decreases
Solution Approach 1:
The patent applies local quality by differentiating cache policies based on individual FAM component characteristics. Each FAM component is analyzed for its access patterns (hotness, sparsity, locality), and cache resources are allocated accordingly - high-access components receive more cache capacity while low-access components receive less, optimizing the balance between memory capacity and access latency.
Solution Approach 2:
The system dynamically adjusts cache allocation policies based on real-time or near-real-time monitoring of data access patterns. The cache manager continuously updates its understanding of which FAM components are most accessed and reallocates cache resources accordingly, transforming the static cache allocation into a dynamic adaptive system that responds to changing workload patterns.
2Device complexity
If cache resources are allocated uniformly to all FAM components, then cache management is simplified, but bandwidth utilization and load balancing deteriorate
Solution Approach 1:
The patent changes the allocation parameters from uniform to differentiated based on measured access characteristics. The cache manager monitors parameters such as access frequency, data hotness, sparsity ratio, and access locality for each FAM component, then uses these parameters to adjust cache allocation decisions, transforming simple uniform allocation into parameter-driven intelligent allocation.
Solution Approach 2:
The system implements feedback mechanisms where the cache manager monitors actual data access patterns and uses this information to adjust cache allocation policies. The flit profiler and pattern statistics analyzer provide continuous feedback about which FAM components are most accessed, enabling the cache manager to iteratively optimize cache distribution to improve bandwidth utilization.
3Quantity of substance
If more FAM components are added to increase memory capacity, then total memory capacity increases, but congestion and access latency increase
Solution Approach 1:
The patent segments the fabric-attached memory into multiple FAM components and applies differentiated cache policies to each segment. By analyzing access patterns at the individual component level, the system can direct cache resources to specific segments that experience high access rates, reducing overall congestion by distributing the caching burden rather than treating all memory components uniformly.
Data Source
AI summary
An example device can include at least one network controller configured to receive a data request and to retrieve data based on the data request, and a cache agent configured to receive a data access parameter based on the data request, and reconfigure a cache for at least one memory cache based on the data access parameter. The data request can be received from a computer device and the data can be retrieved from at least one memory device. An example data access parameter can include a latency of at least one network-attached memory device to retrieve data from the at least one memory device based on the data request. An example device can further comprises a flit profiler configured to determine the data access parameter. Various other methods, systems, and computer-readable media are also disclosed.