Matrix Processing Unit Parallel Dot Product Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computing systems face inefficiencies in performing matrix computations due to limited parallelism and increased processing cycles, particularly in data parallel computing applications like machine learning and neural networks, where existing technologies do not effectively exploit shared dimensions of matrices.
Innovation Solution
The implementation of a Matrix Processing Unit (MPU) with Multiply Accumulate (MACC) Arithmetic Logic Units (ALUs) that split matrices based on shared dimensions, allowing multiple MACC ALUs to compute partial dot products in parallel, and an adder to combine these results efficiently, thereby reducing processing time and enhancing parallelism.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional matrix computation methods are used, then computing accuracy is maintained, but processing time increases and parallelism is limited
Solution Approach 1:
The patent segments the matrix computation task by dividing the right-side matrix into multiple row-split matrices, where each MACC ALU processes a different row-split matrix simultaneously. This segmentation enables parallel computation across multiple ALUs, reducing processing time while maintaining computational accuracy through the accumulation of partial dot products.
2Productivity
If more MACC ALUs are deployed to increase parallelism, then computation speed improves, but device complexity increases
Solution Approach 1:
Each MACC ALU is designed as a universal processing unit that can handle any row-split matrix input. The ALUs share common resources including the shared dimension counter, control logic, and data bus infrastructure. This multi-functionality allows the system to scale parallelism by simply adding identical ALU units rather than designing specialized hardware for each computation path, thereby managing device complexity.
3Productivity
If matrices are split based on shared dimensions, then parallelism is enhanced, but data management complexity increases
Solution Approach 1:
The matrix splitting operation is performed in advance before the computation begins. The right-side matrix is pre-divided into multiple row-split matrices with predetermined dimensions, and the shared dimension is pre-identified. This preliminary action simplifies the data management during computation, as the ALUs receive pre-split data without requiring complex runtime data distribution logic.
Data Source
AI summary
In a method, based on a left side matrix and a right side matrix having a shared dimension, a first Multiply Accumulate Arithmetic Logic Unit (MACC ALU) receives elements of a row of a first column-split matrix and elements of a column of a first column-split matrix. A second MACC ALU receives elements of a row of the second column-split matrix and elements of a column of the second row-split matrix. The first and a second column-split matrices comprise columns of the left side matrix and the first and second row-split matrices comprise rows of the right side matrix. The first and second MACC ALU concurrently compute partial dot products of the column and row elements and the second MACC ALU computes a sum of the partial dot products. A computing system can include the MACC ALUs in a matrix processing unit and can implement the method.