NUMA Coherent Interconnects: Dynamic PoC/PoS Migration for Lower Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In non-uniform memory access (NUMA) systems, constant checks for coherency and serialization between clusters lead to undue traffic and latency, particularly when one cluster is designated as the point-of-coherency (PoC) and point-of-serialization (PoS) but is not actively using the data, while other clusters repeatedly access and update it.
Innovation Solution
A tracker, or snoop filter, dynamically moves the PoC and PoS designations between NUMA clusters based on the state of shared data, using a controller to monitor and manage ownership status, thereby reducing the need for inter-cluster communications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a static PoC and PoS are assigned to one NUMA cluster for shared data, then coherency and serialization are maintained, but interconnect traffic increases and latency increases when another cluster actively accesses the data
Solution Approach 1:
The patent implements dynamic migration of PoC and PoS designations between NUMA clusters based on actual data access patterns. The system monitors which cluster is actively using shared data and relocates the PoC/PoS responsibility to that cluster, transforming the static ownership model into a dynamic one that adapts to changing workload conditions, thereby reducing interconnect traffic and latency
2Loss of energy
If copies of shared data are stored in multiple NUMA clusters, then interconnect traffic is reduced, but coherency checks and serialization requests increase latency
Solution Approach 1:
The patent enables the actively accessing NUMA cluster to serve as its own PoC, performing self-service coherency checks without needing to communicate with other clusters. This eliminates unnecessary interconnect traffic for coherency verification while maintaining data consistency, as the cluster that owns the data locally can verify its own coherency status independently
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system for dynamically controlling point-of-coherency or a point-of-serialization of shared data includes a plurality of processing engines grouped into a plurality of separate clusters and a shared communications path communicatively connecting each of the plurality of clusters to one another. Each respective cluster includes memory shared by the processing engines of the respective cluster, each unit of data in the memory being assigned to a single owner cluster responsible for maintaining an authoritative copy and a single manager cluster permanently responsible for assigning the owner cluster responsibility. Each respective cluster also includes a controller configured to receive data requests, track each of a manager status and an ownership status of the respective cluster, and control ownership status changes with respect to respective units of data based at least in part on the tracked ownership and manager statuses of the respective cluster.