Sparse Matrix Computation Engine for Parallel Vector Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
General-purpose processor circuits, such as CPUs, exhibit low performance and high power consumption when executing numerous computations, particularly in applications like LSTM processing, due to underutilization of matrix multiplication circuits during sparse vector operations.
Innovation Solution
A computation engine that performs parallel computations by packing sparse data vectors into a packed vector, allowing multiple computations to be executed simultaneously, thereby improving system performance and reducing power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If general-purpose processor circuits execute numerous computations, then computations can be performed, but performance is low and power consumption is high
Solution Approach 1:
The computation engine segments the computational workload by separating sparse vector operations from dense matrix operations. It identifies and processes only the non-zero elements of sparse vectors, dividing the computation into essential operations (multiply-and-accumulate for non-zero elements) and unnecessary operations (multiplications involving zero elements), thereby improving performance while reducing energy consumption.
Solution Approach 2:
The patent extracts and removes unnecessary computational operations from the processing pipeline. By detecting zero elements in sparse vectors, the system eliminates multiply-and-accumulate operations that would produce zero results, retaining only the essential computations needed for sparse vector operations, thus enhancing productivity while lowering power consumption.
2Productivity
If matrix multiplication circuits are used for sparse vector operations, then computations can be performed, but the circuits are underutilized
Solution Approach 1:
The computation engine applies local quality by treating different elements of sparse vectors differently based on their values. Non-zero elements receive full computational attention through dedicated multiply-and-accumulate operations, while zero elements are skipped entirely. This localized processing approach optimizes circuit utilization for actual computational needs rather than processing all elements uniformly.
Solution Approach 2:
The system dynamically adapts its operation mode based on the sparsity characteristics of input vectors. By detecting the sparsity pattern and adjusting which circuits remain active and which are deactivated, the system optimizes circuit utilization in real-time, preventing underutilization while maintaining flexibility for different computational workloads.
Data Source
AI summary
A computer system that can execute a sparse matrix instruction is disclosed. The computer system includes a processor circuit that may retrieve, in response to receiving an instruction, an input matrix and a weight matrix. The input matrix may include multiple input vectors, and the weight matrix may include multiple weight vectors, where a number of zero elements in the weight vector exceeds a threshold value. The processor circuit may generate a packed weight vector that includes an orthogonal subset of the plurality of weight vectors. A computation engine may perform, in parallel, a plurality of computations using a subset of the plurality of input vectors corresponding to the subset of the plurality of weight vectors included in the packed weight vector.