Fast Matrix Multiplication via Sparse Decomposition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current matrix multiplication algorithms face challenges in achieving practicality due to large hidden constants and high leading coefficients, making them inefficient for matrices of smaller dimensions despite asymptotic improvements.
Innovation Solution
The implementation of sparse decomposition and alternative basis methods to reduce arithmetic and communication costs, allowing for faster matrix multiplication by transforming matrices into intermediate dimensions and using sparse decompositions, which balance arithmetic and communication costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If fast matrix multiplication algorithms (e.g., Strassen's algorithm) are used to reduce asymptotic complexity, then the leading coefficient increases (e.g., 7n log2 7), making them impractical for smaller matrices
Solution Approach 1:
The patent transforms matrices into intermediate dimensions using alternative basis methods, changing the parameter space in which multiplication occurs. This transformation allows the algorithm to achieve better practical performance by operating in a transformed domain where the effective leading coefficient is reduced, while maintaining the asymptotic complexity benefits of fast algorithms.
Solution Approach 2:
The patent introduces sparse decomposition as an intermediary step between the input matrices and the multiplication operation. By decomposing matrices into sparse components in an intermediate representation, the algorithm reduces the effective arithmetic complexity coefficient while preserving the asymptotic speed advantages, making fast algorithms practical for smaller matrices.
2Productivity
If transformations to intermediate dimensions are applied to reduce arithmetic operations, then communication costs increase
Solution Approach 1:
The patent applies sparse decomposition selectively to different parts of the matrix multiplication process, focusing transformations where they provide the greatest arithmetic efficiency benefit while minimizing communication overhead in other regions. This localized application optimizes the balance between arithmetic operations and communication costs.
Solution Approach 2:
The patent dynamically adjusts the use of alternative basis transformations and sparse decompositions based on matrix size and problem characteristics. For smaller matrices where communication overhead dominates, the algorithm reduces transformation depth; for larger matrices where arithmetic efficiency is more critical, it applies more aggressive transformations, creating a dynamic balance between the two cost types.
Data Source
AI summary
A computer-implemented method comprising: receiving two or more input matrices for a multiplication operation; determining, for each of the input matrices, a series of transformations, and applying the series of transformations respectively to the input matrices to obtain transformed the input matrices, wherein each of the series of transformations reduces a number of arithmetic operations required to perform the multiplication operation, given a desired value of communication costs required to perform the multiplication operation using the computer system, and wherein each of the series of transformations is performed over two or more recursions, wherein at least one of the recursions comprises at least two the transformations; applying a recursive bilinear computation to the transformed two or more input matrices, thereby producing a transformed multiplied matrix; and determining an output series of transformations which are applied to the transformed multiplied matrix, to obtain a product of the input matrices.