Integrated Summation Matrix for Low Latency Tensor Computing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional computing systems face inefficiencies in performing matrix and tensor computations, particularly in machine learning and neural networks, due to serial execution of sum-product and addend matrix operations, which lead to increased computational latency and underutilization of resources.
Innovation Solution
The implementation of an Integrated Summation (ISUM) Tensor Computing System (TCS) that generates ISUM matrices by integrating a matrix addend with multiplicand matrices, allowing for the computation of sum-products without separate addition, thereby reducing latency and improving resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If separate addition of addend matrix to sum-products is performed, then computational accuracy is maintained, but computational latency increases and resource utilization decreases
Solution Approach 1:
The patent merges the addend matrix with the multiplicand matrix by integrating the addend matrix elements into the multiplicand matrix structure. This allows the summation operation to be combined with the matrix multiplication operation, eliminating the need for separate addition steps and reducing computational latency while maintaining accuracy.
Solution Approach 2:
The patent performs preliminary integration of the addend matrix into the multiplicand matrix before the sum-product computation begins. By pre-configuring the integrated matrix structure, the system eliminates the need for subsequent separate addition operations, thereby reducing computational steps and improving throughput.
2Productivity
If separate addition of addend matrix to sum-products is performed, then operational clarity is maintained, but resource utilization decreases
Solution Approach 1:
The patent combines multiple processing stages into a single integrated operation. By merging the addend matrix integration with the sum-product computation pipeline, the system eliminates redundant data transfers and processing steps, thereby improving resource utilization and reducing pipeline complexity.
Solution Approach 2:
The integrated matrix structure serves multiple functions simultaneously: it acts as both the multiplicand for sum-product computation and the container for addend values. This multi-functionality eliminates the need for separate processing pipelines for multiplication and addition, improving resource utilization.
3Productivity
If integrated summation is implemented, then computational efficiency improves, but matrix integration complexity increases
Solution Approach 1:
The patent segments the integrated matrix into distinct regions: multiplicand elements and addend elements. This segmentation allows the system to maintain clear data organization and access patterns while achieving computational efficiency, as each region can be accessed and processed independently during the sum-product operation.
Solution Approach 2:
The patent applies local quality by assigning different functional properties to different parts of the integrated matrix. The multiplicand regions are optimized for multiplication operations while addend regions are optimized for addition operations, allowing each region to be processed with appropriate optimization while maintaining overall computational efficiency.
Data Source
AI summary
A computing method comprises combining an M×K multiplicand matrix and P number of addend vectors to generate an M×(K+P) integrated matrix. The addend vectors can comprise a vector of constants and/or a column of an addend matrix. The method further comprises generating a row-extended matrix comprising a K×N multiplicand matrix and P rows of a constant vector. The method computes (K+P) products of a row of the integrated matrix multiplied by a column of the row-extended matrix and computing an integrated sum of the products. A multiply-accumulate computation can compute the integrated sum and is equivalent to a sum of K number of products of a column of the M×K matrix multiplied by a row of the K×N multiplicand matrix and added to the P number of addend vectors. A computing system can implement the method and can include a matrix computation unit.