Caching Policies Bypassing Sockets to Reduce Probe Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Maintaining cache coherency between processing units on different sockets in NUMA processing systems incurs significant performance and latency costs due to the large number of cache probes transmitted over low-bandwidth socket interfaces, particularly for memory-intensive operations like those performed by GPUs.
Innovation Solution
Implementing a caching policy that selectively bypasses caches on one socket during memory transactions initiated by a processing unit on another socket, allowing memory controllers to directly access memory without sending cache probes over the socket interface, thereby reducing unnecessary cache invalidations and probe transmissions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If cache coherency is maintained between processing units on different sockets, then data consistency is preserved, but performance and latency deteriorate due to numerous cache probes transmitted over low-bandwidth socket interfaces
Solution Approach 1:
The patent segments the memory system into local memory spaces associated with each socket. Each socket has its own local memory that processing units on that socket can access without requiring cache probe coordination with other sockets. This segmentation allows memory transactions to be handled locally without impacting system-wide cache coherency, thereby improving performance while maintaining reliability for cross-socket memory accesses.
2Reliability
If cache probes are transmitted over socket interfaces to maintain cache coherency, then data consistency is ensured, but latency increases due to the time required for probe responses
Solution Approach 1:
The patent extracts the cache probe transmission requirement from local memory transactions by introducing local memory spaces that can be accessed without generating cross-socket cache probes. By taking out the need for inter-socket communication from the memory access path, the system eliminates the latency associated with probe responses while maintaining data consistency through alternative mechanisms for cross-socket coherence.
3Productivity
If memory transactions are directed to local memory resources, then performance is improved, but adaptability deteriorates when local memory is exhausted or inter-thread communication is required
Solution Approach 1:
The patent introduces an intermediary mechanism that manages memory allocation and access across local and remote memory spaces. This intermediary handles the complexity of determining when to use local memory versus remote memory, allowing processing units to automatically access local memory for performance-critical operations while seamlessly falling back to or accessing remote memory when local resources are exhausted or inter-thread communication is needed, thus maintaining both performance and adaptability.
Data Source
AI summary
A processing system includes a first socket, a second socket, and an interface between the first socket and the second socket. A first memory is associated with the first socket and a second memory is associated with the second socket. The processing system also includes a controller for the first memory. The controller is to receive a first request for a first memory transaction with the second memory and perform the first memory transaction along a path that includes the interface and bypasses at least one second cache associated with the second memory.