Vector-Wise Sparsity Pruning Neural Network Weight Matrices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks face inefficiencies in execution and increased latency due to their complexity, with existing sparsity techniques offering poor speedup or accuracy, particularly in sparse neural networks, and hardware such as CPUs and GPUs remaining inefficient.
Innovation Solution
The implementation of vector-wise sparsity in neural networks by dividing matrices into vectors, selecting non-zero elements to form a pruned matrix, and using this pruned matrix for execution, which allows for improved speed and power efficiency by reducing mathematical operations and memory transfers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing sparsity techniques are applied to neural networks, then execution efficiency is improved, but accuracy deteriorates
Solution Approach 1:
The patent segments the weight matrix into multiple vectors and applies sparsity independently to each vector. This allows selective pruning of zero elements at the vector level while preserving important weight patterns, thereby improving execution efficiency without significantly degrading accuracy compared to traditional element-wise sparsity methods.
Solution Approach 2:
The patent applies different sparsity patterns to different vectors based on their local characteristics. By analyzing the distribution of non-zero elements in each vector and applying targeted pruning strategies, the method maintains local weight quality where it matters most while achieving overall sparsity for efficiency gains.
2Speed
If existing sparsity techniques are applied to neural networks, then speedup is achieved, but workload irregularity increases
Solution Approach 1:
By segmenting the matrix into vectors and applying uniform vector-wise sparsity operations, the patent creates more regular workload patterns compared to irregular element-wise sparsity. Each vector can be processed independently with predictable patterns, improving hardware utilization and reducing the negative effects of workload irregularity on execution speed.
3Power
If matrix size is increased in neural networks, then computational power is improved, but execution latency increases
Solution Approach 1:
The patent extracts and removes zero elements from the weight matrix at the vector level, creating a pruned matrix with fewer elements to process. This reduction in the number of mathematical operations directly decreases execution latency while preserving the essential computational power needed for accurate neural network inference.
Data Source
AI summary
The present disclosure relates to systems and methods for providing vector-wise sparsity in neural networks. In some embodiments, an exemplary method for providing vector-wise sparsity in a neural network, comprises: dividing a matrix associated with the neural network into a plurality of vectors; selecting a first subset of non-zero elements from the plurality of vectors to form a pruned matrix; and outputting the pruned matrix for executing the neural network using the pruned matrix.