Parallel MPI Matrix Transpose via Block Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Parallel MPI matrix transposition in distributed-memory multiprocessor environments is inefficient due to high compute time and memory usage, particularly in in-place transpositions, where temporary copies of data buffers are required for data movement between processors.
Innovation Solution
The method involves dividing the input matrix into submatrix data blocks of a predetermined size and using MPI APIs to send and receive these blocks without making an in-memory copy, allowing for in-place transposition without the need for temporary buffers, thereby reducing compute time and memory consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If temporary copy of storage media buffer is created for in-place transpose, then data can be moved between processors simultaneously, but compute time and memory space increase
Solution Approach 1:
The patent segments the storage media buffer into multiple blocks and processes them in a specific order. By dividing the buffer into blocks that can be independently sent and received, the system enables overlapping of send and receive operations for different blocks, thereby hiding communication latency and reducing overall compute time without requiring a full temporary copy of the buffer
Solution Approach 2:
The patent performs preliminary actions by sending certain blocks before receiving their corresponding transposed blocks. This allows the system to overlap communication operations and utilize memory bandwidth more efficiently, reducing the critical path of the transpose operation without compromising data correctness
2Reliability
If temporary copy of storage media buffer is created for in-place transpose, then data can be moved between processors simultaneously, but memory space consumption increases
Solution Approach 1:
The patent divides the storage media buffer into multiple manageable blocks rather than creating a full temporary copy. This segmentation allows the system to reuse portions of the original buffer for receive operations after send operations are complete, significantly reducing peak memory space requirements while maintaining data movement correctness through proper block ordering and synchronization
Solution Approach 2:
The patent makes the storage media buffer serve multiple functions: it acts as both the send buffer and receive buffer at different times and for different blocks. This multi-functionality eliminates the need for a separate temporary copy, reducing memory space consumption while ensuring data correctness through careful management of buffer usage states
Data Source
AI summary
Computer-implemented techniques for fast block-based parallel message passing interface (MPI) transpose are disclosed. The techniques achieve an in-place parallel matrix transpose of an input matrix in a distributed-memory multiprocessor environment with reduced consumption of computer processing time and storage media resources. An in-memory copy of the input matrix or a submatrix thereof to use as the send buffer for MPI send operations is not needed. Instead, by dividing the input matrix in-place into data blocks having up to at most a predetermined size and sending the corresponding data block(s) for a given submatrix using an MPI API before receiving any data block(s) for the given submatrix using an MPI API in the place of the sent data block(s), making the in-memory copy to use a send buffer can be avoided and yet the input matrix can be transposed in-place.