Hybrid Memory Module Bridge Network for Parallel Cache Transfers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The input/output (I/O) bandwidth of memory modules has not kept pace with the increasing internal bandwidth of memory devices, and multi-drop buses used to enhance memory capacity lead to inefficient data transfer due to shared bandwidth and latency issues, particularly when one memory module acts as a cache for another.
Innovation Solution
Implementing a hybrid memory module bridge network with multiple interfaces, including a wide, low-latency interface for host-memory communication and narrow, high-frequency, high-latency interfaces for inter-memory data transfers, allowing cache line fills and writebacks to occur in parallel with host requests without interfering with cache demand accesses, and using serializer/deserializer ports for efficient data transfer between memory modules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple memory modules are connected using a single channel to increase memory capacity, then memory capacity is improved, but bandwidth efficiency deteriorates due to shared bandwidth and frequent bus turnarounds
Solution Approach 1:
The patent divides the memory system into multiple independent channels, each capable of handling memory operations autonomously. This segmentation allows different memory modules to access different channels simultaneously, eliminating the bandwidth bottleneck of shared single-channel access and enabling parallel data transfers that improve overall bandwidth efficiency while maintaining increased memory capacity.
Solution Approach 2:
The patent transitions from a single-dimensional shared bus architecture to a multi-dimensional channel-based architecture. By adding the channel dimension, memory modules can access data through multiple independent pathways simultaneously, transforming the bottlenecked single-path access into parallel multi-path access, thereby improving bandwidth efficiency without sacrificing memory capacity.
2Speed
If a wide, low-latency interface is used for host-memory communication, then external bus performance is improved, but internal bandwidth capacity is underutilized
Solution Approach 1:
The patent introduces channel controllers as intermediary components between the wide host-memory interface and the internal memory arrays. These controllers manage and distribute data flows, enabling the system to handle both high-speed external transfers and intensive internal bandwidth utilization simultaneously by coordinating access patterns and optimizing data routing through the channel architecture.
Solution Approach 2:
The patent implements dynamic resource allocation within the channel architecture, where bandwidth and access priorities are adjusted based on real-time operational demands. This allows the system to optimize for external bus performance when host transfers are active, while seamlessly switching to maximize internal bandwidth utilization during cache operations, thereby resolving the contradiction between the two performance metrics.
3Adaptability or versatility
If cache operations are performed on a shared bus, then cache functionality is achieved, but latency increases due to bus turnarounds and protocol overhead
Solution Approach 1:
The patent segments cache operations into dedicated channel transactions, separating cache hits and misses from general memory bus traffic. This allows cache operations to be handled through direct channel access rather than competing for shared bus bandwidth, significantly reducing latency while preserving full cache functionality through targeted data retrieval paths.
Solution Approach 2:
The patent enables continuous cache operations by maintaining persistent channel connections and eliminating bus turnaround delays. Cache data transfers proceed continuously through dedicated channels without the interruptions and protocol overhead inherent in shared bus architectures, thereby reducing latency while sustaining efficient cache functionality.
Data Source
AI summary
Systems, apparatuses, and methods for implementing a hybrid memory module bridge network and buffers are disclosed. A system includes one or more host processors and multiple memory modules. Each memory module includes a relatively low pin count, high-bandwidth serial link to one or more other memory modules to perform inter-memory data transfers without consuming host-memory bandwidth. In one embodiment, a first memory module acts as a cache and a second memory module acts as the main memory for the system. The traffic between the host and the first memory module utilizes a first interface, and the cache traffic between the first and second memory modules utilizes a second interface. Cache line fill and writeback transfers between the first and second memory modules occur in parallel with timing-critical cache demand accesses from the host, in a latency-tolerant and buffered manner, without interfering with the cache demand accesses.