Memory Write Request Grouping for Response Bandwidth Efficiency

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing computing systems face limitations in bandwidth due to inefficiencies in response command bandwidth, even when data path bandwidth to system memory is high, as current techniques do not effectively manage response command bandwidth.

Innovation Solution

A computing system with multiple clients and vertically stacked memory dies uses a communication fabric to group memory write requests and reduce response bandwidth by sending a single acknowledgment for multiple write requests, thereby optimizing data transmission and power consumption.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If multiple individual acknowledgment responses are sent for each memory write request, then clients can reliably track completion of each write operation, but response command bandwidth is consumed and power is wasted

Engineering Contradiction:
Improvewrite completion trackingVSAvoidresponse command bandwidth
Core Design Contradiction:
ReliabilityVSProductivity

Solution Approach 1:

The patent combines multiple individual acknowledgment responses into a single batched acknowledgment response. The memory controller groups multiple write requests that target the same memory bank and sends one consolidated acknowledgment response for the entire group, rather than sending separate responses for each write request. This merging approach maintains reliability by ensuring all clients receive confirmation of their write operations while dramatically reducing response command bandwidth consumption.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The patent uses a copy of the write request information to generate the acknowledgment response. Instead of requiring individual response transactions, the memory controller creates an acknowledgment response that copies and confirms the status of multiple write requests in a single communication event. This copying mechanism allows efficient bandwidth utilization while preserving the ability to track each individual write operation's completion status.

Inventive Principle:
Principle #26Copying

2Reliability

If multiple individual acknowledgment responses are sent for each memory write request, then clients receive timely confirmation of each write operation, but power consumption increases due to frequent communication

Engineering Contradiction:
Improvewrite completion confirmationVSAvoidpower consumption
Core Design Contradiction:
ReliabilityVSLoss of energy

Solution Approach 1:

The patent merges multiple acknowledgment responses into a single batched response, reducing the frequency of communication events between the memory controller and clients. By grouping multiple write requests and sending one consolidated acknowledgment, the system maintains reliable write completion confirmation while significantly reducing power consumption associated with frequent signal transmissions and receiver activation.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The patent implements periodic batching of acknowledgment responses rather than continuous individual responses. The memory controller accumulates write requests over a defined period or until a threshold is reached, then sends a single acknowledgment response for the accumulated group. This periodic action pattern reduces power consumption by creating intervals of reduced communication activity while still providing timely confirmation of write operations.

Inventive Principle:
Principle #19Periodic action

3Productivity

If write requests are grouped and batched for processing, then response command bandwidth efficiency improves, but system complexity increases due to grouping logic and state tracking

Engineering Contradiction:
Improveresponse command bandwidth efficiencyVSAvoidgrouping and tracking logic
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent introduces an intermediary batching mechanism within the memory controller that mediates between individual write requests and acknowledgment responses. This intermediary component automatically groups write requests based on simple criteria (such as targeting the same memory bank) and manages the state tracking required for batched processing. By localizing the complexity to this intermediary layer, the system achieves improved bandwidth efficiency while keeping the added complexity contained and manageable within the memory controller's existing architecture.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS10684965B2Method to reduce write responses to improve bandwidth and efficiency
Publication Date: 2020.06.16 ADVANCED MICRO DEVICES INC
  • US10684965B2 patent drawing
  • US10684965B2 patent drawing
  • US10684965B2 patent drawing

AI summary

Systems, apparatuses, and methods for routing traffic between clients and system memory are disclosed. A computing system includes system memory and one or more clients, each capable of generating memory access requests. The computing system also includes a communication fabric for transferring traffic between the clients and the system memory. The fabric includes master units for interfacing with clients and grouping write requests with a same target together. The fabric also includes slave units for interfacing with memory controllers and for sending a single write response when each write request in a group has been serviced. When the master unit receives the single write response for the group, it sends a respective acknowledgment response for each of the multiple write requests in the group to clients that generated the multiple write requests.