GPU Cluster Cache Coherency via Shared Memory Bus

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing system architectures face area and power constraints, leading to the need for a large number of Peer-to-Peer (P2P) high-speed buses between GPUs, which increases quadratically with the number of GPUs, resulting in bottlenecks during peak performance due to the inability to support the required number of buses.

Innovation Solution

A GPU cluster architecture with multiple GPU nodes, each having a local cache and shared memory, maintains cache coherency through an internal bus, reducing the need for multiple P2P high-speed buses by allowing data to be shared efficiently across nodes within a cluster.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If one P2P high speed bus connection is used between each GPU pair, then data communication bandwidth between GPUs is improved, but the number of required buses increases quadratically as the number of GPUs increases

Engineering Contradiction:

Improvedata communication bandwidthVSAvoidnumber of P2P high speed buses

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

Multiple GPU nodes share a common P2P bus instead of having dedicated point-to-point connections between each pair. The bus is divided into multiple channels that can be accessed by different GPU nodes simultaneously, reducing the total number of buses from n*(n-1)/2 to a linear scale with the number of GPUs.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent introduces a shared memory space as an additional dimension for data exchange between GPUs. Instead of direct point-to-point connections, data can be published to shared memory and subscribed to by multiple GPUs, transforming the communication topology from a complete graph to a shared resource model.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Productivity

If the number of P2P high speed buses is increased to support more GPUs, then data communication capability is improved, but area and power constraints are exceeded

Engineering Contradiction:

Improvedata communication capabilityVSAvoidarea consumption

Core Design Contradiction:

ProductivityVSArea of stationary object

Solution Approach 1:

Inventive Principle:

Principle #5Merging (Combining)

3Area of stationary object

If lower performance buses are used to connect GPU pairs due to area and power constraints, then area and power consumption are reduced, but performance becomes bottlenecked during peak periods

Engineering Contradiction:

Improvearea consumptionVSAvoidperformance

Core Design Contradiction:

Area of stationary objectVSProductivity

Solution Approach 1:

The shared P2P bus is divided into multiple independent channels that can operate simultaneously. This segmentation allows the bus to provide aggregate bandwidth equivalent to multiple dedicated connections while using a single physical bus infrastructure, thereby achieving high performance without requiring proportional increases in area and power.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11144459B2Cache coherency adopted GPU shared memory

Publication Date: 2021.10.12 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

An approach is provided in which a system includes a GPU cluster. The GPU cluster includes multiple GPU nodes, that each includes a GPU core and a corresponding local cache. The GPU cluster also includes a shared memory and an internal bus that maintains cache coherency between the shared memory and the local caches included in the GPU nodes.