Merging Memory Access Responses in Cache Hierarchy

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current computing systems face limitations in memory access efficiency due to inefficiencies in the response bandwidth of lower memory hierarchy levels, leading to restricted data bandwidth and performance issues in data-intensive applications.

Innovation Solution

Implementing a computing system with multiple compute units and a communication fabric that merges memory access requests from multiple compute units into a single response, allowing the system to reduce bandwidth utilization by transmitting merged responses through the fabric, thereby enhancing memory access efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If multiple individual memory access responses are transmitted through the communication fabric, then each compute unit receives its required data, but the bandwidth utilization is high and system performance is limited

Engineering Contradiction:

Improvememory access efficiencyVSAvoidnumber of responses transmitted

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The patent merges multiple individual memory access responses into a single consolidated response message that travels through the communication fabric. This consolidation reduces the number of separate transmissions required, thereby decreasing bandwidth utilization and improving overall memory access efficiency while ensuring all compute units receive their required data

Inventive Principle:

Principle #5Merging (Combining)

2Speed

If techniques are used to saturate the available bandwidth for accessing lower levels of the memory hierarchy, then data transfer rate increases, but the overall bandwidth is still limited due to inefficiencies in response bandwidth

Engineering Contradiction:

Improvedata transfer rateVSAvoidoverall bandwidth

Core Design Contradiction:

SpeedVSProductivity

Solution Approach 1:

By consolidating multiple response transmissions into a single merged response, the patent eliminates redundant data transfers and communication overhead. This merging approach optimizes the utilization of response bandwidth, allowing the system to achieve higher overall bandwidth efficiency without sacrificing data transfer rate to lower memory levels

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS11403221B2Memory access response merging in a memory hierarchy

Publication Date: 2022.08.02 ADVANCED MICRO DEVICES INC

AI summary

A system and method for efficiently processing memory requests are described. A computing system includes multiple compute units, multiple caches of a memory hierarchy and a communication fabric. A compute unit generates a memory access request that misses in a higher level cache, which sends a miss request to a lower level shared cache. During servicing of the miss request, the lower level cache merges identification information of multiple memory access requests targeting a same cache line from multiple compute units into a merged memory access response. The lower level shared cache continues to insert information into the merged memory access response until the lower level shared cache is ready to issue the merged memory access response. An intermediate router in the communication fabric broadcasts the merged memory access response into multiple memory access responses to send to corresponding compute units.