Hardware Accelerator Partial Matrix Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Matrix multiplication operations in machine learning model training are computationally intensive and inefficient, especially when dealing with large matrices, as they require significant processing resources and time.
Innovation Solution
A hardware accelerator is designed to perform matrix multiplication by dividing large matrices into smaller submatrices, computing intermediate partial matrices in a front-end processing area, and summing them in a back-end area to produce partial matrices, which are then output, allowing for parallelization and increased computation speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If matrix multiplication is performed using conventional processing methods, then the computation can be completed, but the processing time and computational resources required are excessive
Solution Approach 1:
The patent divides large matrices into smaller submatrices and further segments the computation into multiple iterations. Each iteration processes a portion of the matrix multiplication, allowing parallel execution and reducing the time required for each individual computation step while maintaining the ability to compute the full result by combining partial results.
Solution Approach 2:
The patent introduces a temporal dimension by performing matrix multiplication across multiple iterations rather than attempting to complete the entire computation in a single step. This multi-iteration approach allows the system to process different portions of the matrix multiplication in parallel, effectively adding a time dimension to the computation strategy.
2Reliability
If large matrices are processed in their entirety, then complete results are obtained, but memory and bandwidth usage become excessive
Solution Approach 1:
The patent segments large matrices into smaller submatrices that can be processed individually in each iteration. This segmentation reduces the amount of data that needs to be held in memory at any given time, as only the necessary submatrices for the current iteration are loaded, while still producing accurate complete results by combining the partial results from all iterations.
Solution Approach 2:
The patent performs partial matrix multiplications in each iteration, computing only the necessary portion of the result for that iteration rather than attempting to compute the entire result at once. This partial action approach reduces memory requirements while ensuring that the complete accurate result is obtained through accumulation of partial results across multiple iterations.
Data Source
AI summary
A computing device, including a hardware accelerator configured to receive a first matrix and receive a second matrix. The hardware accelerator may, for a plurality of partial matrix regions, in a first iteration, read a first submatrix of the first matrix and a second submatrix of the second matrix into a front-end processing area. The hardware accelerator may multiply the first submatrix by the second submatrix to compute a first intermediate partial matrix. In each of one or more subsequent iterations, the hardware accelerator may read an additional submatrix into the front end processing area. The hardware accelerator may compute an additional intermediate partial matrix as a product of the additional submatrix and a submatrix reused from an immediately prior iteration. The hardware accelerator may compute each partial matrix as a sum of two or more of the intermediate partial matrices and may output the plurality of partial matrices.