Shared Cache Stashing for Lower NUMA Read Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In non-uniform memory access (NUMA) systems, memory access performance is degraded due to varying access rates based on the location of targeted memory, especially as the number of chips increases, leading to potential bottlenecks and reduced performance.
Innovation Solution
A system on chip (SoC) architecture that includes a shared system level cache and cache controller to transmit data proactively to remote chips after an atomic operation, using a stash value mechanism to determine when to send data based on feedback from the remote chip.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If chips access memory through buses in UMA systems, then all chips have equal access rates to memory, but as the number of chips increases, bottlenecks arise leading to reduced memory access performance
Solution Approach 1:
The patent divides the unified memory system into distributed memory modules across multiple chips. Each chip has its own local memory, segmenting the previously centralized memory resource. This segmentation eliminates the bottleneck of shared bus access by allowing parallel memory operations across chips, thereby maintaining reliability through structured access protocols while improving overall productivity.
Solution Approach 2:
The patent transitions from a single-dimension unified memory access model to a multi-dimensional hierarchical memory architecture. Memory is organized across spatial dimensions (different chips) and access levels (local vs. remote access), creating a NUMA (Non-Uniform Memory Access) structure. This dimensional change allows simultaneous access from multiple chips without bottlenecking a single bus, resolving the contradiction between equal access rates and performance.
2Adaptability or versatility
If chips have their own internal memories in NUMA systems, then memory access flexibility increases, but access rates vary based on memory location causing performance degradation
Solution Approach 1:
The patent implements a cache coherence protocol that performs preliminary actions to maintain data consistency across distributed memories. When data is accessed or modified on one chip, the system proactively notifies other chips that may have cached copies, ensuring they update or invalidate their copies. This preliminary action eliminates the performance penalty of location-based access variations by keeping data synchronized across the distributed memory hierarchy, maintaining both flexibility and productivity.
Solution Approach 2:
The patent employs feedback mechanisms through cache coherence protocols where chips report memory access patterns and data validity status to the system. This feedback allows the memory management unit to optimize data placement and access routing dynamically. By incorporating feedback from distributed memory accesses, the system adapts to maintain consistent performance regardless of memory location, resolving the contradiction between access flexibility and performance uniformity.
3Loss of time
If data is transmitted to remote chips in NUMA systems, then read latency is reduced, but data consistency and synchronization become more complex
Solution Approach 1:
The patent introduces a cache controller as an intermediary between processors and the distributed memory system. This intermediary manages data transmission, validity tracking, and consistency protocols automatically. When data needs to be transmitted to remote chips, the cache controller handles the transmission and updates coherence status without requiring complex application-level synchronization code. This intermediary layer reduces read latency by optimizing data paths while managing consistency complexity internally, resolving the contradiction between speed and complexity.
Data Source
AI summary
A system on chip including: a plurality of processors; a system level cache memory that is shared by the plurality of processors; and a cache controller that transmits first data to another chip if an atomic operation has been performed on the first data, when the system level cache memory receives the first data from at least one of the plurality of processors.