Communication Control Program for DBCSR Matrix Multiplication

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

The Cannon matrix product algorithm for DBCSR matrices experiences performance degradation due to increased communication overhead when the number of processes exceeds the matrix size, leading to inefficiencies in matrix multiplication, particularly under strong scaling conditions.

Innovation Solution

Implementing a communication control program that utilizes a multidimensional torus structure for interconnecting nodes, allowing bidirectional communication between processes, and optimizing communication buffers through a hash table to reduce the number of communication steps and improve network utilization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If the number of processes is increased beyond matrix size, then parallel processing capability is improved, but communication overhead increases and performance degrades

Engineering Contradiction:
Improveparallel processing capabilityVSAvoidcommunication overhead
Core Design Contradiction:
ProductivityVSLoss of time

Solution Approach 1:

The patent transitions from a conventional one-dimensional or two-dimensional process grid to a multidimensional torus structure (3D, 4D, or higher dimensions). This dimensional expansion allows processes to be organized in multiple spatial dimensions, enabling more efficient communication patterns when the number of processes exceeds the matrix size. The torus topology provides wraparound connections in each dimension, reducing communication distance and overhead in high-process configurations.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The patent segments the matrix multiplication algorithm into distinct communication phases (forward communication, backward communication, and local computation) that can be executed in an optimized sequence. By dividing the computation into blocks and processes across multiple dimensions, the system can manage communication more efficiently, reducing the total communication overhead even when using many processes.

Inventive Principle:
Principle #1Segmentation

2Productivity

If bidirectional communication is implemented, then network utilization is improved, but communication complexity increases

Engineering Contradiction:
Improvenetwork utilizationVSAvoidcommunication complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent implements bidirectional communication in a periodic, structured manner using the torus topology. Communication occurs in alternating phases (forward and backward) along different dimensions, with each phase following a predetermined pattern. This periodic structure allows the system to achieve high network utilization while maintaining manageable complexity through regular, repeating communication templates rather than arbitrary communication patterns.

Inventive Principle:
Principle #19Periodic action

Solution Approach 2:

The patent performs preliminary setup of communication patterns and buffer allocations before the actual matrix multiplication begins. Communication routes, buffer sizes, and phase timing are predetermined based on the torus topology and process configuration, allowing the system to execute bidirectional communication efficiently during runtime without dynamically determining complex communication patterns on-the-fly.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS20240012874A1Computer-readable recording medium storing communication control program, information processing apparatus, and communication control method
Publication Date: 2024.01.11 FUJITSU LTD
  • US20240012874A1 patent drawing
  • US20240012874A1 patent drawing
  • US20240012874A1 patent drawing

AI summary

A non-transitory computer-readable recording medium records a communication control program for causing a computer to execute a processing of: processing, by a plurality of information processing devices intercoupled by a multidimensional torus structure, blocks of a matrix in Distributed Block Compressed Sparse Row (DBCSR) format in a plurality of processes in a distributed manner; and communicating the blocks in both directions for each of a left matrix and a right matrix of the matrix at each stage of a matrix product algorithm.