HBM Command Scheduling With Deterministic and Estimated PIM Latency

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing high-bandwidth memory (HBM) systems face challenges in efficiently handling deterministic and non-deterministic processing-in-memory (PIM) operations due to varying latency times, which affect energy efficiency and latency in data-intensive applications like deep neural networks and high-performance computing.

Innovation Solution

A quasi-synchronous interface protocol for HBM systems that incorporates both deterministic and non-deterministic latency times, using a one-step HBM+ protocol for simple PIM operations and a two-step HBM+ protocol for complex operations, with the logic circuit converting host commands into PIM commands and providing time estimates for the host device.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If deterministic latency protocol is used for simple PIM operations, then processing speed is improved, but system complexity increases due to protocol differentiation

Engineering Contradiction:

Improveprocessing speedVSAvoidprotocol complexity

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

The system dynamically selects between deterministic and non-deterministic protocols based on operation complexity. The logic circuit detects whether a PIM operation is simple or complex and automatically applies the appropriate protocol, allowing the system to optimize for speed when possible while maintaining flexibility for complex operations.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The protocol system is segmented into two distinct modes: deterministic latency protocol for simple operations and non-deterministic protocol for complex operations. This segmentation allows each protocol to be optimized for its specific use case without compromising the other, resolving the contradiction between speed and complexity.

Inventive Principle:

Principle #1Segmentation

2Adaptability or versatility

If non-deterministic latency protocol is used for complex PIM operations, then protocol flexibility is improved, but processing speed deteriorates

Engineering Contradiction:

Improveprotocol flexibilityVSAvoidprocessing speed

Core Design Contradiction:

Adaptability or versatilityVSSpeed

Solution Approach 1:

The system dynamically adapts its protocol selection based on the specific PIM operation requirements. For complex operations requiring flexibility, the non-deterministic protocol is applied, while for simpler operations, the deterministic protocol provides faster processing. This dynamic adaptation resolves the contradiction between flexibility and speed.

Inventive Principle:

Principle #15Dynamics

3Use of energy by moving object

If PIM operations are scheduled with varying latency times, then energy efficiency is improved, but latency predictability worsens

Engineering Contradiction:

Improveenergy efficiencyVSAvoidlatency predictability

Core Design Contradiction:

Use of energy by moving objectVSLoss of time

Solution Approach 1:

The latency model is segmented into deterministic and non-deterministic portions. The deterministic portion provides predictable timing for critical operations, while the non-deterministic portion allows flexibility for energy-efficient scheduling of less time-critical operations. This segmentation resolves the contradiction between energy efficiency and latency predictability.

Inventive Principle:

Principle #1Segmentation

4Reliability

If host device waits for PIM operation completion, then operation accuracy is ensured, but productivity deteriorates

Engineering Contradiction:

Improveoperation accuracyVSAvoidhost productivity

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The logic circuit performs preliminary processing of host commands into PIM commands before the host needs to wait for completion. This preliminary action allows the host to continue with other productive tasks while the PIM operations execute in the background, resolving the contradiction between ensuring operation completion and maintaining host productivity.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback mechanisms where the logic circuit notifies the host when PIM operations are complete or when estimated latency periods expire. This feedback allows the host to resume operations based on actual completion status rather than waiting passively, improving productivity while maintaining reliability through proper synchronization.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS12524153B2Host-based and client-based command scheduling in large bandwidth memory systems

Publication Date: 2026.01.13 SAMSUNG ELECTRONICS CO LTD

AI summary

A high-bandwidth memory (HBM) system includes an HBM device and a logic circuit. The logic circuit receives a first command from the host device and converts the received first command to a processing-in-memory (PIM) command that is sent to the HBM device through the second interface. A time between when the first command is received from the host device and when the HBM system is ready to receive another command from the host device is deterministic. The logic circuit further receives a fourth command and a fifth command from the host device. The fifth command requests time-estimate information relating to a time between when the fifth command is received and when the HBM system is ready to receive another command from the host device. The time-estimate information includes a deterministic period of time and an estimated period of time for a non-deterministic period of time.