Private Last-Level Cache Segmentation for Thread Local Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Commercial CPUs with shared last-level caches experience increased latency and power consumption due to multiple hops required to access data private to a single core, which is exacerbated in data center environments with thousands of CPUs operating continuously.
Innovation Solution
Implementing a CPU architecture with a shared and private portion in the last-level cache, where the CPU core determines if a data access request is for Thread Local Storage (TLS) type data and routes it to a private cache for TLS requests, reducing interconnect traffic and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a shared last-level cache is used among multiple CPU cores, then the entire cache capacity can be shared by multiple cores, but it increases data access latency and power consumption due to multiple hops required to access data private to a single core
Solution Approach 1:
The last-level cache is segmented into a shared portion and a private portion. The private portion is exclusively assigned to a specific CPU core, while the shared portion is accessible by multiple cores. This segmentation allows TLS data to be accessed locally without requiring hops across the interconnect network, reducing latency while maintaining cache sharing capabilities for non-TLS data.
Solution Approach 2:
The patent applies local quality by providing each CPU core with a dedicated private portion of the last-level cache for storing and accessing Thread Local Storage (TLS) data. This private cache portion is located locally at each core, eliminating the need for remote access through the interconnect network. Meanwhile, the shared portion maintains the ability to serve multiple cores, thus optimizing both local and shared access patterns.
2Adaptability or versatility
If a shared last-level cache is used among multiple CPU cores, then the entire cache capacity can be shared by multiple cores, but it increases power consumption due to multiple hops required to access data
Solution Approach 1:
The last-level cache is divided into private and shared portions, with the private portion locally attached to each CPU core. This segmentation ensures that TLS data accesses are served locally without traversing the power-consuming interconnect network, while the shared portion continues to provide caching services for non-TLS data across multiple cores.
Solution Approach 2:
By providing a local private cache portion at each CPU core, the patent eliminates the need for remote cache accesses via the interconnect network for TLS data. This local access mechanism significantly reduces power consumption associated with signal transmission across the chip, while the shared portion maintains system-wide cache sharing functionality.
3Adaptability or versatility
If TLS data is stored in a distributed manner across multiple LLC slices attached to different cores, then the entire LLC capacity is shared, but it causes increased latency due to multiple hops to fetch data
Solution Approach 1:
The last-level cache is segmented into private slices and shared slices. The private slice at each core is dedicated to storing TLS data for that core, ensuring that TLS data remains localized. This segmentation prevents the need to fetch TLS data from remote slices through the interconnect network, thereby maintaining high data access speed while still allowing the shared slices to be utilized by multiple cores for non-TLS data.
Solution Approach 2:
The patent implements local quality by associating a private cache slice with each CPU core specifically for TLS data storage and access. This local placement ensures that TLS data can be accessed at full cache speed without the overhead of interconnect hops. The shared slices continue to provide capacity sharing across cores for data that benefits from multi-core accessibility.
Data Source
AI summary
A multi-core CPU includes a Last-Level Cache (LLC) interconnected with a plurality of cores. The LLC may include a shared portion and a private portion. The shared portion may be shared by the plurality of cores. The private portion may be connected to a first core of the plurality of cores and may be exclusively assigned to the first core. The first core may be configured to initiate a data access request to access data stored in the LLC and initiate a data access request to access data stored in the LLC. The first core may route the data access request to the private portion based on the determination that the data access request is the TLS type of access request and route the data access request to the shared portion based on the determination that the data access request is not the TLS type of access request.