Sparse Matrix Multiplication Pseudo-Space Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current algorithms for sparse matrix-matrix multiplication, such as SRUMMA, face scalability limitations due to O(√P) communication steps and logistic complexity, hindering the processing of large-scale sparse matrices despite increasing computational power.
Innovation Solution
The method employs a pseudo-space approach with pseudo particles to map matrix elements into a pseudo-space, segmenting this space among processes and using a midpoint method to assign and transmit matrix elements for efficient multiplication, reducing inter-process communication and improving scalability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If SRUMMA algorithm is used for sparse matrix multiplication, then the multiplication can be performed using existing dense matrix algorithms, but the communication steps and logistic complexity scale as O(√P) which prevents linear scaling with increasing number of processes
Solution Approach 1:
The patent divides the computation domain into distinct segments (inner products, outer products, result accumulation) that can be independently assigned to different process groups. This segmentation allows each segment to be optimized separately and enables linear scaling by distributing segments across increasing numbers of processes without the O(√P) bottleneck of traditional algorithms.
Solution Approach 2:
The patent introduces a new computational dimension by separating the multiplication into independent inner product computations and outer product accumulations. This dimensional separation allows processes to work independently on inner products while simultaneously accumulating results, breaking the sequential dependency that causes O(√P) scaling in traditional algorithms.
2Reliability
If O(√P) communication steps are used in sparse matrix multiplication, then the algorithm can handle moderate-sized matrices, but the communication overhead prevents processing of large-scale matrices even with increased computational power
Solution Approach 1:
The patent performs preliminary computation of inner products in advance before the outer product accumulation phase. By pre-computing the inner products and storing them, the algorithm eliminates the need for repeated communication during the accumulation phase, enabling large-scale matrix processing with reduced communication overhead.
Solution Approach 2:
The patent introduces intermediate storage structures for inner product results that act as mediators between the computation phase and accumulation phase. These intermediaries allow results to be cached and reused without requiring continuous inter-process communication, thereby enabling processing of larger matrices with fewer communication steps.
3Device complexity
If inter-process communication does not decrease linearly with increasing number of processes, then the algorithm maintains simplicity, but the parallel behavior prevents efficient utilization of additional computational resources
Solution Approach 1:
The patent implements dynamic process assignment where processes can be dynamically allocated to different computation segments based on the available hardware resources. This dynamic allocation allows the algorithm to automatically adapt to the number of available processes, maintaining simplicity while achieving linear resource utilization efficiency through flexible process management.
Data Source
AI summary
Embodiments include performing sparse matrix-matrix multiplication. Aspects include receiving a first matrix and a second matrix, providing a pseudo-space for the first and second matrices, and defining pseudo-space segments and assigning the pseudo-space segments to certain processes. Aspects also include assigning matrix elements of the first and second matrix to pseudo-space segments using a midpoint method thereby assigning the matrix elements to processes associated with the pseudo-space segments, assigning a result matrix element of a result matrix to a pseudo-space segment using a midpoint method thereby assigning the result matrix element to a further process associated with the pseudo-space segment and transmitting matrix elements of the first and second matrix required to establish a result matrix element to the further process which processes the result matrix element. Aspects further include performing a multiplication procedure by the further process based on the received matrix elements of the first and second matrix.