Systolic Array Refactoring for Uneven Matrix Dimensions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Systolic arrays face inefficiencies when handling input matrices that cannot be evenly split into sub-matrices matching the number of processing elements, leading to increased complexity and cost in hardware solutions, particularly when using a large number of processing elements.
Innovation Solution
The method involves refactoring the original problem by converting the input matrix into a new matrix with dimensions that can be evenly divided into the systolic array, using new parameters for the extended portion to solve the problem in integer-sized portions, thereby reducing the computational burden and logic requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the input matrix is split into sub-matrices matching the systolic array size, then the problem can be solved using the systolic array, but additional processing is needed when the matrix cannot be split evenly, increasing hardware complexity
Solution Approach 1:
The patent divides the input matrix into multiple sub-matrices that can be processed by the systolic array. When the matrix dimensions are not evenly divisible by the number of processing elements, the patent segments the problem into complete sub-matrices and a remainder portion, processing them separately to avoid hardware complexity while maintaining full problem-solving capability
Solution Approach 2:
The patent introduces a controller as an intermediary component that manages the processing of sub-matrices and remainder portions. The controller coordinates data flow between the systolic array and memory, handling the complexity of uneven matrix splitting without requiring complex hardware modifications to the systolic array itself
2Productivity
If results are calculated at each processing element for uneven matrix splitting, then the problem is solved, but the circuit needs to calculate and store results at each PE, which is burdensome
Solution Approach 1:
The patent extracts the remainder portion processing from the main systolic array computation flow. Instead of having every processing element calculate and store results for unevenly sized inputs, the patent separates the complete sub-matrix processing (handled by PEs) from the remainder portion processing (handled separately), reducing the computational burden on individual PEs
Solution Approach 2:
The patent performs preliminary processing by dividing the input matrix into sub-matrices before feeding them to the systolic array. This preliminary segmentation allows the systolic array to operate efficiently on evenly-sized blocks, with the remainder portion handled in a separate preliminary preparation step, avoiding unnecessary calculations at each PE
3Ease of operation
If a large multiplexer is used to select the last row data from all PEs, then the actual result can be selected, but this necessitates a large multiplexer and additional routing complexity
Solution Approach 1:
The patent extracts the result selection logic from the systolic array output stage. Instead of using a large multiplexer to select from all PE outputs, the patent separates the result selection into a post-processing step that operates on aggregated data, eliminating the need for complex routing and large multiplexers within the systolic array hardware
Solution Approach 2:
The patent changes the dimensionality of the result selection approach by moving from a spatial selection method (large multiplexer selecting from multiple PE outputs) to a temporal/aggregated selection method. The results are accumulated over time through the systolic array's natural data flow, and the final result is obtained through simpler aggregation logic rather than complex spatial multiplexing
Data Source
AI summary
Methods and systems for refactoring a problem include refactoring an original problem having a dimension that cannot be broken into an integer number of portions, each portion having a number of problem elements equal to a size of a systolic array, into a new problem having a dimension that can be broken into an integer number of portions, each portion having a number of problem elements equal to the size of the systolic array. The new problem is solved with the systolic array. The systolic array has a size defined by an integer number of processing elements and is configured to solve portions of problems having a number of problem elements equal to the number of processing elements.