Speculative Data Delivery in Multiprocessor Memory Hierarchies

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In multiprocessor data processing systems, data access latency is high due to the need for systemwide coherence responses before data can be sourced from caches or memory, limiting the efficiency of read-type memory access requests.

Innovation Solution

Enabling system memory or lower level caches to source data speculatively before receiving a systemwide coherence response, and allowing multiple memory devices to act as data sources for read-type memory access requests, thereby reducing latency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If a systemwide coherence response is required before data can be sourced from caches or memory, then cache coherency is maintained, but data access latency increases

Engineering Contradiction:

Improvecache coherencyVSAvoiddata access latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by allowing caches to source data speculatively before receiving the systemwide coherence response. The coherence protocol is executed in the background while data transfer proceeds in parallel, eliminating the sequential dependency that causes latency.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system maintains continuous useful action by allowing data transfers to proceed without interruption while the coherence protocol completes. Multiple memory devices can act as data sources simultaneously, ensuring that data access operations continue throughout the coherence response time.

Inventive Principle:

Principle #20Continuity of useful action

2Device complexity

If only a single data source is allowed for each read-type memory access request, then coherence control is simplified, but data access efficiency decreases

Engineering Contradiction:

Improvecoherence control complexityVSAvoiddata access efficiency

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

The system segments the data source selection by allowing multiple memory devices to act as independent data sources for the same read-type memory access request. Each memory device can independently provide data without requiring centralized coordination, dividing the data source function across multiple independent units.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system applies universality by enabling multiple memory devices to universally serve as data sources for read-type memory access requests. Any memory device that has the requested data can act as a data source, making the data source role flexible and multi-functional rather than fixed to a single device.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11615024B2Speculative delivery of data from a lower level of a memory hierarchy in a data processing system

Publication Date: 2023.03.28 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A multiprocessor data processing system includes multiple vertical cache hierarchies supporting a plurality of processor cores, a system memory, and an interconnect fabric coupled to the system memory and the multiple vertical cache hierarchies. Based on a request of a requesting processor core among the plurality of processor cores, a master in the multiprocessor data processing system issues, via the interconnect fabric, a read-type memory access request. The master receives via the interconnect fabric at least one beat of conditional data issued speculatively on the interconnect fabric by a controller of the system memory prior to receipt by the controller of a systemwide coherence response for the read-type memory access request. The master forwards the at least one beat of conditional data to the requesting processor core.