Hybrid Hardware-Software Cache Coherency for NUMA Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multi-processor systems face challenges in cache coherence, where scaling memory capacity independently of processor numbers is costly and inefficient, with hardware-managed cache coherence consuming bandwidth and increasing latency, while software-managed coherence requires complex programming and performs poorly under high protocol events.
Innovation Solution
A hybrid cache-coherent NUMA system that optimally separates software-managed and hardware-managed cache coherency, allowing independent scaling of memory and processors, using a network of communication links to interconnect nodes with node controllers that maintain cache state and ownership information, and switch between hardware and software protocols based on memory access requests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If hardware-managed cache coherence protocols are used, then cache coherence is maintained, but memory bandwidth is consumed and latency increases
Solution Approach 1:
The patent segments the cache coherence management into two distinct parts: hardware-managed coherence for processor-attached memory and software-managed coherence for node-attached memory. This segmentation allows each type of memory to use the most appropriate coherence management approach, reducing overall hardware bandwidth consumption while maintaining coherence where critical.
Solution Approach 2:
The system dynamically selects between hardware-managed and software-managed coherence protocols based on the memory access request destination. The cache coherence management logic determines whether to use hardware or software management on a per-request basis, optimizing performance and bandwidth usage adaptively.
2Quantity of substance
If the number of memory devices is increased to accommodate digital content, then memory capacity increases, but system cost increases
Solution Approach 1:
The patent divides memory into two categories: processor-attached memory and node-attached memory. This segmentation enables independent scaling of memory capacity without proportionally increasing processor count or system complexity. Node controllers can manage large amounts of memory attached to individual nodes, allowing memory to be added independently of processor requirements.
Solution Approach 2:
The patent introduces a new dimension of memory organization by attaching memory directly to nodes rather than only to processors. This creates a two-dimensional memory hierarchy (processor-attached and node-attached) that allows independent scaling of memory capacity without linearly increasing processor count or system cost.
3Loss of energy
If software-managed cache coherence protocols are used, then hardware bandwidth is reduced, but programming complexity increases
Solution Approach 1:
The patent introduces cache coherence management logic as an intermediary layer between hardware and software. This logic automatically manages the selection between hardware-managed and software-managed coherence protocols, shielding application programmers from the complexity of coherence management while still enabling software-managed coherence where appropriate.
4Productivity
If both the number of processors and memory devices are increased, then computing power and memory capacity increase, but system cost increases
Solution Approach 1:
The patent segments memory management at the node level, with each node having its own controller that can independently manage processor-attached and node-attached memory. This allows memory capacity to be increased independently of processor count, enabling systems to scale memory without proportionally increasing processor count or system cost.
Solution Approach 2:
The patent creates a hierarchical memory architecture with two levels: processor-attached memory and node-attached memory. This two-dimensional organization allows independent scaling of memory capacity without linearly increasing processor count, enabling cost-effective memory expansion while maintaining computing power.
Data Source
AI summary
A node controller is provided to include a first interface to interface with one or more processors, a second interface including a plurality of ports to interface with node controllers within a base node and other nodes in the cache-coherent interconnect network. The node controller can further include a third interface to interface with a first plurality of memory devices and a cache coherence management logic. The cache coherence management logic can maintain, based on a first circuitry, hardware-managed cache coherency in the cache-coherent interconnect network. The cache coherence management logic can further facilitate, based on a second circuitry, software-managed cache coherency in the cache-coherent interconnect network.