Sparse Patch Embedding With Activation Filtering for Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Performing matrix operations, such as multiplication, on dense matrices in neural networks requires significant bandwidth and processing time, necessitating a reduction in bandwidth and processing requirements.
Innovation Solution
Implementing sparse activation filtering to modify neural networks, particularly vision transformers, by applying sparsity constraints to input tensors and images, allowing the use of matrix multiplication acceleration units to perform sparse patch embedding and reduce computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If dense matrix multiplication is used in neural networks, then computational accuracy is maintained, but bandwidth consumption and processing time increase significantly
Solution Approach 1:
The patent applies sparsity constraints to transform dense activation tensors into sparse tensors by setting certain activation values to zero based on magnitude thresholds. This parameter change in the data representation enables matrix multiplication acceleration units to skip computations involving zero values, thereby reducing bandwidth consumption and processing time while maintaining acceptable computational accuracy.
2Quantity of substance
If sparse activation filtering is applied to reduce bandwidth, then memory utilization decreases, but computational complexity increases
Solution Approach 1:
The patent performs sparse activation filtering as a preliminary step before matrix multiplication operations. By pre-processing activation tensors to identify and zero out insignificant values based on sparsity constraints, the system prepares data in a format that enables faster computation. This preliminary action reduces the quantity of data requiring subsequent processing while the computational complexity is managed through efficient identification algorithms.
Data Source
AI summary
Apparatuses, systems, and methods to enable matrix multiplication acceleration by modifying an input to apply sparsity through sparse activation filtering. In at least one embodiment, a neural network modifies pixels within an image through sparse activation filtering to enable use of one or more matrix multiplication acceleration units to perform a sparse patch embedding operation.