Sub-numa Clustering for Cache Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As the number of cores on a chip multiprocessor (CMP) increases, the average distance between cores and cache slices grows, leading to higher latency during cache access due to the shared and distributed cache structure, necessitating efficient management of stored/cached data.
Innovation Solution
The solution involves grouping cores and their associated cache slices into clusters, partitioning memory space into address regions, and assigning memory controllers to manage data flow, allowing cache access to be biased towards the closest cluster, thereby reducing latency and improving data locality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the number of cores on a CMP increases to accomplish more tasks in parallel, then productivity increases, but the average distance between cores and cache slices increases leading to higher cache access latency
Solution Approach 1:
The patent divides the unified cache structure into multiple cache clusters, where each cluster is physically closer to a subset of cores. This segmentation reduces the average distance between cores and their designated cache slices, thereby reducing cache access latency while maintaining high parallel processing capability through the distributed cluster architecture.
Solution Approach 2:
The patent implements non-uniform memory access (NUMA) architecture where different cache clusters have different access characteristics for different cores. Cores have preferential access to nearby cache clusters, creating local optimization where frequently accessed data resides in geographically closer cache memory, reducing access time for the majority of operations.
2Adaptability or versatility
If a shared and distributed cache structure is used to serve multiple cores, then adaptability increases, but the distance between cores and cache slices increases resulting in higher latency
Solution Approach 1:
The cache is segmented into multiple distributed clusters rather than a single unified structure. Each cluster serves a specific group of cores, maintaining the distributed nature for adaptability while reducing the physical access distance within each cluster, thereby lowering latency.
Solution Approach 2:
The patent introduces a hierarchical dimension to the cache architecture, with multiple levels of caching (L1, L2, L3) organized in clusters. This dimensional organization allows cores to access frequently used data in nearby lower-level caches while maintaining access to the full memory space through higher-level caches, balancing adaptability and speed.
Data Source
AI summary
Method and apparatus to efficiently store and cache data. Cores of a processor and cache slices co-located with the cores may be grouped into a cluster. A memory space may be partitioned into address regions. The cluster may be associated with an address region from the address regions. Each memory address of the address region may be mapped to one or more of the cache slices grouped into the cluster. A cache access from one or more of the cores grouped into the cluster may be biased to the address region based on the association of the cluster with the address region.