Systolic Array ODP Circuits for Sparse Matrix Multiplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processing architectures struggle to efficiently handle the computational demands of contemporary applications like deep learning algorithms and large-scale data analytics, facing inefficiencies in data movement and parallel processing, and lack flexibility and scalability across diverse computational tasks.
Innovation Solution
A systolic array that operates in two modes based on sparsity in input matrices, performing matrix multiplication efficiently by using sub-processing element arrays and output direct path circuits, allowing for dense and sparse matrix operations with reduced clock cycles and enhanced processing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a conventional processing architecture is used for matrix multiplication, then the system can handle general computational tasks, but the processing speed and computational efficiency deteriorate due to inefficiencies in data movement and parallel processing
Solution Approach 1:
The processing array is divided into multiple sub-processing element arrays arranged in a pipeline structure. Each sub-PE array handles a specific stage of matrix multiplication, enabling parallel processing across multiple segments while maintaining overall computational efficiency. This segmentation resolves the contradiction by organizing complexity into manageable, parallelizable units.
Solution Approach 2:
The patent introduces a temporal dimension through pipeline staging, where different sub-PE arrays operate at different time stages. Data flows through multiple spatial stages (sub-PE arrays) in a coordinated manner, transforming a single-dimensional sequential process into a multi-dimensional parallel pipeline architecture that improves computational throughput.
2Loss of time
If all operations are performed on input matrix elements including zeros, then the processing is straightforward, but the processing time increases due to unnecessary operations on zero values
Solution Approach 1:
The patent extracts and removes zero elements from the input matrix before processing. By identifying and eliminating zero values in advance, the system avoids performing unnecessary multiplication and accumulation operations on these elements, thereby reducing processing time without significantly complicating the overall operation flow.
Solution Approach 2:
Instead of performing complete matrix multiplication on all elements, the patent applies partial action by processing only the non-zero elements of the input matrix. This selective processing approach reduces the total number of operations required, directly addressing the time loss issue while maintaining computational accuracy.
3Adaptability or versatility
If the systolic array operates in a single mode, then the design is simpler, but the adaptability to different input matrix types (dense and sparse) deteriorates
Solution Approach 1:
The patent designs the processing array with universal sub-PE arrays that can function in multiple modes. The same hardware structure adapts to handle both dense matrices (where all elements are processed) and sparse matrices (where zero elements are skipped), eliminating the need for completely separate circuit configurations for different input types.
Solution Approach 2:
The system dynamically switches between different operational modes based on the characteristics of the input matrix. Control logic dynamically adjusts the behavior of sub-PE arrays and data flow patterns to match the input type, enabling the fixed hardware to exhibit dynamic adaptability without requiring multiple static configurations.
4Use of energy by moving object
If data is continuously moved between processing elements, then the computation can proceed, but the power consumption increases due to frequent data movement operations
Solution Approach 1:
The patent performs preliminary actions by pre-processing the input matrix to remove zero elements before they enter the processing pipeline. This upfront preparation reduces the volume of data that needs to be moved through the system during computation, thereby lowering power consumption associated with data movement while maintaining computational speed.
Data Source
AI summary
Provided is a systolic array including a plurality of sub processing element (PE) arrays, wherein each of the plurality of sub PE arrays is configured to receive an input signal and to generate an output signal based on the received input signal, and a plurality of output direct path (ODP) circuits is configured to operate in a first mode and in a second mode, wherein an ODP circuit of the plurality of ODP circuits receives the output signal of a previous sub PE array of the plurality of sub PE arrays, wherein the ODP circuit provides the output signal to a next sub PE array of the plurality of sub PE arrays in the first mode, and wherein the ODP circuit outputs the output signal in the second mode.