Sparsified Neural Network Layers for Semi-Digital Crossbar Architectures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hardware accelerators for machine learning applications do not realize significant power savings in analog or semi-analog crossbar networks due to high overhead from powering up arrays, even with a large fraction of zero-valued elements in pruned networks, as the zero-valued elements are not locally correlated.
Innovation Solution
A method that maps a binary matrix to an undirected graph form, applies a two-way graph partition algorithm to minimize edge cuts, and uses a greedy algorithm to find row or column permutations that maximize the transfer of non-zero elements from sparse blocks to non-sparse blocks, thereby sparsifying or densifying the matrix.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If pruning methods are applied to sparsify neural network layers, then the fraction of zero-valued elements increases, but power savings are not realized in analog or semi-analog crossbar networks due to overhead from powering up arrays
Solution Approach 1:
The patent applies local quality by transforming the global sparsity pattern into local block sparsity. Through graph partitioning and greedy algorithms, the method creates densely packed zero blocks that can be locally processed. This allows analog crossbar networks to exploit sparsity at the block level rather than requiring global sparsity, enabling power savings by keeping only necessary compute blocks active while powering down others.
Solution Approach 2:
The patent segments the neural network weight matrix into multiple blocks through graph partitioning. This segmentation transforms a globally sparse matrix into a block-structured matrix where zero elements are concentrated in specific blocks. The segmentation enables selective activation of compute blocks in analog crossbar networks, reducing power consumption by deactivating blocks with predominantly zero values.
2Productivity
If analog or semi-analog crossbar networks are used for matrix vector multiplication, then computation efficiency is improved, but overhead from powering up arrays prevents significant power savings even with pruned networks
Solution Approach 1:
The patent introduces dynamics by enabling selective activation and deactivation of compute blocks based on the block sparsity pattern. Rather than statically powering up entire arrays, the system dynamically activates only the necessary blocks containing non-zero elements, allowing analog crossbar networks to adapt power consumption to the actual computational workload while maintaining high computation efficiency.
3Loss of substance
If zero-valued elements are not locally correlated, then sparsity exists but power savings cannot be achieved in analog crossbar architectures due to array powering overhead
Solution Approach 1:
The patent applies preliminary action by pre-processing the weight matrix through graph partitioning and greedy algorithms to create a block-sparsed structure before deployment to analog crossbar networks. This preliminary transformation organizes zero-valued elements into locally correlated blocks, enabling the hardware to exploit sparsity patterns in advance and activate only necessary compute blocks, thereby eliminating power overhead while preserving sparsity benefits.
Data Source
AI summary
A method and system are provided. The method includes mapping a binary matrix to an undirected graph form, applying a two-way graph partition algorithm to the mapped binary matrix that minimizes edge cuts between partitions in the mapped binary matrix, applying a greedy algorithm recursively to find a set of row or column permutations that maximizes a transfer of non-zeros from sparse blocks to nonsparse blocks, and sparsifying or densifying the binary matrix according to the applied greedy algorithm.