Packetized Memory Routing for Balanced Multi-Processor Bandwidth

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional memory systems suffer from imbalanced system bandwidth and complexity due to limitations in memory protocols, traditional memory subsystem architectures, and processor-specific memory access models, leading to suboptimal performance in multi-processor systems.

Innovation Solution

Implementing a Hybrid Memory Cube (HMC) interconnect system with packetized memory links and a logic base that enables balanced system bandwidth and reduced design complexity by using a hierarchical and parallel approach, allowing for efficient communication between multiple processors and memory domains through packetized requests and responses.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If conventional memory protocols and traditional memory subsystem architectures are used, then system compatibility and ease of implementation are maintained, but system bandwidth balance and performance are reduced

Engineering Contradiction:

Improvesystem bandwidthVSAvoidmemory subsystem architecture complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent segments the memory subsystem into multiple independent memory domains, each managed by its own memory controller. This segmentation allows each domain to operate independently with optimized bandwidth, resolving the contradiction by enabling high system bandwidth through parallel operation of multiple domains while keeping each domain's complexity manageable.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a new dimension of organization by creating a hierarchical memory architecture with multiple levels of memory domains and controllers. This dimensional change allows the system to achieve balanced bandwidth across multiple processors while distributing complexity across the hierarchical structure rather than concentrating it in a single controller.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Productivity

If Memory Control Hubs are used to enhance memory performance, then local memory bandwidth is improved, but system interconnect bandwidth remains insufficient creating imbalance

Engineering Contradiction:

Improvelocal memory bandwidthVSAvoidsystem interconnect balance

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The patent makes the memory interconnect universal by designing it to handle both local memory traffic and system-wide interprocessor communication through the same packetized link structure. This multi-functionality allows the interconnect to provide balanced bandwidth for both local and system-wide operations, resolving the imbalance between local memory bandwidth and system interconnect bandwidth.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Adaptability or versatility

If private processor communication buses are implemented for multi-processor communication, then processor communication capability is enhanced, but system complexity and design difficulty increase

Engineering Contradiction:

Improveprocessor communication capabilityVSAvoidsystem interconnect complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent merges the memory interconnect and processor communication bus into a single unified packetized link structure. This consolidation allows the system to achieve enhanced processor communication capability while reducing overall system complexity, as the same physical infrastructure serves both memory access and interprocessor communication functions.

Inventive Principle:

Principle #5Merging (Combining)

4Productivity

If high bandwidth memory devices are used, then local memory access bandwidth is increased, but system interconnect bandwidth becomes a bottleneck providing only a small fraction of this bandwidth

Engineering Contradiction:

Improvelocal memory access bandwidthVSAvoidsystem interconnect bandwidth balance

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent segments the system into multiple memory domains with distributed memory controllers, allowing local memory access bandwidth to be high within each domain while system interconnect bandwidth is aggregated across multiple domains. This segmentation resolves the bottleneck by enabling parallel communication paths through the interconnect.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements preliminary action by having memory controllers pre-process and packetize memory requests before transmission over the interconnect. This preparation allows the interconnect to efficiently handle high bandwidth traffic from multiple high-performance memory devices without becoming a bottleneck.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12511041B2Packet routing between memory devices and related apparatuses, methods, and memory systems

Publication Date: 2025.12.30 MICRON TECHNOLOGY INC

AI summary

An interconnect system includes host devices, one or more memory devices, and a routing system to connect the host devices and the one or more memory devices. Respective ones of the host devices include an interface to communicate packet requests over respective packetized links. Respective ones of the one or more memory devices include an interface to receive and respond to the packet requests over the respective packetized links. The routing system includes devices interconnected in a routing topology. Respective ones of the devices include a switch and interfaces. The routing system is to route the packet requests and responses between the host devices and respective memory device destinations over the respective packetized links.