Shared Cache Slice Mapping for Lower Tail Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As the number of processor cores and shared cache slices increase, tail latency and bandwidth challenges arise due to increased distances and unscaled interconnects, leading to inefficient cache access and higher latency.
Innovation Solution
Implement selectable slice mapping that dynamically assigns sections of physical address space to cache slices based on latency tolerance and memory usage, grouping slices closer to processor cores to reduce interconnect bandwidth and improve performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the number of processor cores and shared cache slices increases, then the need for external memory access decreases, but tail latency increases due to increased distance between core and farthest slice
Solution Approach 1:
The shared cache is divided into multiple slices, and the patent applies selective mapping to assign different address spaces to different slices. This segmentation allows each slice to be optimally positioned relative to processor cores, reducing the maximum distance any data must travel while maintaining the benefits of having multiple cache slices for reduced external memory access.
Solution Approach 2:
The patent implements non-uniform mapping where different regions of the address space are mapped to different cache slices based on their access patterns and latency requirements. This local quality approach ensures that frequently accessed data with strict latency requirements are placed in slices closer to processor cores, while less time-sensitive data can reside in farther slices.
2Reliability
If the number of slices in shared cache increases, then external memory access is reduced, but interconnect bandwidth requirements become more challenging to meet
Solution Approach 1:
By segmenting the address space and mapping it selectively to cache slices, the patent reduces the number of interconnect paths that need to handle all traffic. Each slice receives only the traffic relevant to its mapped address space, distributing the bandwidth burden more evenly and reducing peak bandwidth requirements across the interconnect fabric.
Solution Approach 2:
The selective mapping approach enables dynamic allocation of address spaces to cache slices based on workload characteristics. This dynamic behavior allows the system to adapt to changing bandwidth requirements by reconfiguring which address ranges map to which slices, optimizing interconnect utilization under different operating conditions.
Data Source
AI summary
Systems and techniques for selectable slice mapping in shared cache levels are described. In one example, a processor includes a cache system having a shared cache level of a hierarchy of cache levels and slice hashing circuitry associated with the shared cache level. The shared cache level includes multiple slices accessible by threads running on multiple processor cores. The slice hashing circuitry assigns memory addresses used by a particular thread to a subset of the multiple slices closest to the processor core on which the thread runs. The assignment of the slice subset is based on the latency requirements or the data usage of the thread in at least one implementation. The described techniques improve tail latencies for multiple core systems and alleviate the need for additional interconnections for shared cache levels.