Sparse Data Processing Chip Using Outer-Product Matrix Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks, particularly Transformer networks, suffer from sparsification issues leading to low computing performance, high resource demands, and low resource utilization due to the presence of zero values in weight matrices and feature maps, which are not optimally handled by current hardware.
Innovation Solution
A data processing chip and method utilizing the outer product method for matrix multiplication operations, which leverages the sparse characteristics of weight matrices to compress data, reducing bandwidth requirements and improving resource utilization by eliminating zero-value weights and aggregating valid data into fewer sub-objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional hardware processes sparse data with zero values, then data processing can be performed, but computing performance is low and resource utilization is poor
Solution Approach 1:
The patent extracts and removes zero-value weights from the weight matrix before processing. By identifying and eliminating zero values in advance, the system only processes non-zero elements, thereby improving computing performance and reducing resource waste on unnecessary calculations.
Solution Approach 2:
The patent performs preliminary compression of the weight matrix to generate a compressed weight matrix before the main processing operation. This preliminary action of removing zero values and compressing the matrix structure prepares the data in advance, enabling more efficient processing and better resource utilization during execution.
2Reliability
If zero values are retained in weight matrices and feature maps, then data完整性 is maintained, but storage and transmission resource requirements increase
Solution Approach 1:
The patent extracts zero values from the weight matrix and feature maps, separating them from the meaningful data. By removing these redundant zero elements, the system reduces storage and transmission requirements while maintaining the integrity of non-zero data through selective preservation and indexing mechanisms.
Solution Approach 2:
The patent changes the representation parameters of the data by transforming the dense weight matrix into a compressed format with different structural parameters. This parameter change enables more efficient storage and transmission by encoding only necessary information while reconstructing the original data structure when needed.
3Reliability
If all data sub-objects are processed individually, then processing completeness is achieved, but the number of processing operations is excessive
Solution Approach 1:
The patent merges multiple data sub-objects by aggregating their valid (non-zero) elements into fewer target data sub-objects. This merging reduces the total number of processing operations required while maintaining processing completeness by ensuring all valid data from original sub-objects is preserved and correctly assembled in the compressed representation.
Data Source
AI summary
A data processing chip includes hardware processing channels configured to obtain a first target data set formed by one or more pieces of first target data included in at least one first target data sub-object, obtain one or more pieces of second data included in a second data object corresponding to the target data processing channel, perform matching on the one or more pieces of first target data and the one or more pieces of second data according to first position information corresponding to each piece of first target data and second position information corresponding to each piece of second data to obtain matched data that includes one or more pieces of first target data and one or more pieces of second data that matching each other, and perform data processing on the matched data.