Shared Cache Memory Controller With Virtual Channel Arbitration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multi-core systems, memory access latency and throughput are bottlenecked by the coherence interconnect, which handles accesses serially to maintain data consistency, leading to CPU stall overhead and reduced shared memory throughput due to varying cache structures and access types among components.
Innovation Solution
A multi-core shared memory controller (MSMC) with integrated address translation and cache prewarming functionality, utilizing a data routing unit (DRU) that translates virtual memory addresses to physical addresses and supports DMA operations directly, reducing latency by preloading data into caches and optimizing memory access through virtual channels and arbitration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the coherence interconnect handles memory accesses serially to maintain data consistency, then data coherency is guaranteed, but memory access latency increases and throughput decreases
Solution Approach 1:
The patent segments the memory access arbitration into multiple virtual channels (VCs), where each VC can independently arbitrate and forward memory requests. This segmentation allows parallel processing of multiple memory accesses across different VCs, maintaining data coherency within each VC while enabling concurrent operations across VCs, thus improving throughput without sacrificing reliability
Solution Approach 2:
The patent implements dynamic arbitration mechanisms where the interconnect can adaptively allocate bandwidth and prioritize requests based on current system conditions, cache states, and coherency requirements. This dynamic approach allows the system to optimize for either coherency or throughput depending on the operational context, resolving the static contradiction between these two requirements
2Reliability
If the coherence interconnect handles memory accesses serially to guarantee atomicity, then data consistency is maintained, but CPU stall overhead increases
Solution Approach 1:
The patent implements cache prewarming functionality that proactively loads data into caches before it is actually needed by the CPU. By performing this action in advance, the system reduces the likelihood of cache misses that would cause CPU stalls, thereby reducing time loss without compromising data consistency through the pre-established coherency protocols
3Adaptability or versatility
If different components have different cache structures and access types, then system heterogeneity is supported, but the interconnect becomes a bottleneck
Solution Approach 1:
The patent designs the interconnect with universal virtual channel infrastructure that can handle multiple access types (read, write, allocate, invalidate) and support heterogeneous components (CPUs, GPUs, DSPs, NPU) through a unified arbitration framework. This multi-functional design allows the same interconnect structure to serve diverse components with different cache configurations without requiring component-specific arbitration logic, managing complexity while maintaining versatility
4Loss of time
If cache components issue cache allocate accesses with critical line first request, then CPU stall overhead is reduced, but shared memory throughput decreases when multiple CPUs compete
Solution Approach 1:
The patent applies local quality by allowing each virtual channel to independently prioritize and handle critical line requests from its associated CPUs, while the overall system maintains balanced throughput through distributed arbitration. This localized prioritization within VCs reduces individual CPU stall overhead without creating system-wide bottlenecks, as other VCs can simultaneously service other CPUs' memory requests
Data Source
AI summary
Techniques for accessing memory by a memory controller, comprising receiving, by the memory controller, a memory management command to perform a memory management operation at a virtual memory address, translating the virtual memory address to a physical memory address, wherein the physical memory address comprises an address within a cache memory, and outputting an instruction to the cache memory based on the memory management command and the physical memory address.