Low Power Parallel Matrix Multiply Pipeline
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Matrix multiply operations on stream processors are power-intensive and time-consuming, necessitating improvements in performance, power consumption, and latency.
Innovation Solution
A low power parallel matrix multiply pipeline is implemented using multiple vector register files and execution pipelines with dot product units that perform dot product and outer product operations, allowing for overlapping execution and reduced memory bandwidth, thereby optimizing matrix multiply operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If matrix multiply operations are performed on stream processors using conventional sequential methods, then the operations can be completed with simple architecture, but the power consumption is high and the execution time is long
Solution Approach 1:
The matrix multiply operation is divided into multiple independent dot product units that operate in parallel. Each dot product unit processes a specific element of the result matrix independently, allowing the computation to be segmented into simultaneous operations rather than sequential execution, thereby reducing overall power consumption while maintaining computational completeness
Solution Approach 2:
The patent transitions from sequential single-pipeline execution to parallel multi-pipeline execution by adding a temporal dimension to the computation. Multiple dot product units operate simultaneously on different data elements, effectively moving the computation from a one-dimensional sequential process to a multi-dimensional parallel process, which reduces power consumption per operation
2Productivity
If conventional matrix multiply operations are used on stream processors, then the implementation is simple, but the latency is high and throughput is low
Solution Approach 1:
The computation is segmented into multiple dot product units that can operate simultaneously. Each unit is responsible for computing a specific element of the result matrix, allowing multiple computations to proceed in parallel rather than sequentially, thereby increasing throughput and reducing latency
Solution Approach 2:
The patent implements continuous data flow through the pipeline where input data is continuously fed into multiple dot product units, and results are continuously generated and written back to register files. This continuous operation eliminates idle cycles and maintains productive action throughout the computation process, maximizing throughput
3Productivity
If multiple dot product units operate in parallel, then throughput increases, but data movement and memory bandwidth requirements increase
Solution Approach 1:
Input data is pre-loaded into vector register files before the parallel computation begins. This preliminary action allows multiple dot product units to access their required operands from fast on-chip memory rather than requiring continuous access to external memory, thereby reducing the overall data movement burden while maintaining high throughput
Solution Approach 2:
Vector register files serve as intermediary storage between external memory and the parallel dot product units. These intermediaries buffer the data, allowing parallel units to operate continuously without directly competing for external memory bandwidth, thus enabling high throughput with controlled data movement
Data Source
AI summary
Systems, apparatuses, and methods for implementing a low power parallel matrix multiply pipeline are disclosed. In one embodiment, a system includes at least first and second vector register files coupled to a matrix multiply pipeline. The matrix multiply pipeline comprises a plurality of dot product units. The dot product units are configured to calculate dot or outer products for first and second sets of operands retrieved from the first vector register file. The results of the dot or outer product operations are written back to the second vector register file. The second vector register file provides the results from the previous dot or outer product operations as inputs to subsequent dot or outer product operations. The dot product units receive the results from previous phases of the matrix multiply operation and accumulate these previous dot or outer product results with the current dot or outer product results.