In-Situ Sparse Matrix Expansion for Faster Array Loading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing matrix multiply units face inefficiencies in loading input matrices, particularly sparse matrices, due to the inclusion of many zero values, which prolongs the time required for matrix operations and inefficient utilization of memory resources.
Innovation Solution
A matrix computation unit with comparator units in each cell determines whether to store or shift non-zero input values based on index metadata, allowing only non-zero values to be loaded into the array, reducing the number of clock cycles needed for matrix injection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If all input values including zero values are loaded into the matrix multiply unit, then the matrix multiplication can be performed, but the loading time increases and throughput decreases
Solution Approach 1:
The patent extracts and removes zero values from the input matrix before loading into the matrix multiply unit. By identifying and eliminating null elements, the system only loads non-zero values along with their metadata, significantly reducing the number of values that need to be loaded and stored, thereby improving throughput and reducing loading time.
Solution Approach 2:
The patent performs preliminary processing of the input matrix by pre-computing and storing metadata that indicates the locations of non-zero values. This preliminary action allows the system to quickly identify and load only the necessary non-zero values during matrix multiplication, avoiding the time-consuming process of loading and then skipping zero values.
2Productivity
If all input values including zero values are loaded into the matrix multiply unit, then the complete matrix is processed, but memory utilization becomes inefficient
Solution Approach 1:
The patent extracts zero values from the input matrix and excludes them from the loading process. By loading only non-zero values and their corresponding metadata, the system optimizes memory utilization and reduces the energy required for memory operations, as fewer memory writes and storage operations are performed.
3Ease of operation
If index metadata is sent to each cell to determine storage location, then non-zero values can be correctly positioned, but the device complexity increases
Solution Approach 1:
The patent introduces metadata as an intermediary that carries location information for non-zero values. This metadata acts as a mediator between the compressed input data and the matrix multiply unit, enabling correct positioning of values without requiring complex addressing logic in each cell. The metadata simplifies the overall system by providing explicit location information that cells can directly use.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for loading a matrix into a circuit having an array having M×N cells. One of the methods includes: receiving a plurality of non-zero input values from a first input matrix; receiving index metadata that indicates, for each non-zero input value in the plurality of input values, which cell of the M×N cells in the array the non-zero input value should be loaded into; sending the non-zero input values and the index metadata to the M×N cells; and at a particular cell of the M×N cells in the array: receiving a particular non-zero input value and corresponding index metadata; and determining from the corresponding index metadata for the particular non-zero input value whether to store the particular non-zero input value at the cell or to shift the particular non-zero input value to another cell.