Memristive Accelerator Sparse Weight Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Sparse weight matrices in artificial neural networks lead to inefficient inference processes due to trivial computations and high energy consumption, as they contain a large number of zero-value coefficients that result in multiplying or adding zeros during matrix-vector multiplication operations.
Innovation Solution
Partitioning the sparse weight matrix into sub-blocks, where one sub-block contains only zero-value weights and another contains non-zero value weights, and assigning non-zero weights to clusters of memristor crossbars, while cutting off power to clusters without non-zero weights to reduce energy consumption and improve performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If sparse weight matrices are used in artificial neural networks, then the network can represent complex patterns with fewer parameters, but the inference process becomes inefficient due to trivial computations involving zero-value coefficients
Solution Approach 1:
The patent segments the weight matrix into sparse blocks (containing zero-value coefficients) and dense blocks (containing non-zero coefficients). This segmentation allows the system to process only the dense blocks during inference, eliminating trivial computations involving zeros while maintaining the parameter efficiency of sparse representations.
Solution Approach 2:
The patent applies partial action by computing only the necessary dense blocks rather than performing full matrix-vector multiplication. This selective computation approach processes only the relevant portions of the weight matrix that contribute to the output, avoiding wasteful operations on zero-value coefficients.
2Stability of the object's composition
If sparse weight matrices with zero-value coefficients are processed, then the network maintains its sparse structure, but energy consumption increases due to performing trivial computations such as multiplying by zeros
Solution Approach 1:
The patent extracts and identifies zero-value coefficient blocks from the weight matrix, separating them from the computational path. By taking out these sparse blocks and representing them efficiently without performing actual computations on them, the system preserves the sparse structure while eliminating the energy-wasting trivial multiplications.
Solution Approach 2:
The system performs partial computation by selectively processing only dense blocks and skipping sparse blocks entirely. This approach maintains the mathematical correctness of the sparse matrix multiplication while avoiding the energy consumption associated with computing operations on zero-value coefficients.
3Adaptability or versatility
If all clusters in the circuit are kept active to handle potential non-zero weights, then the system maintains flexibility for weight assignment, but energy consumption increases due to powering unused clusters
Solution Approach 1:
The patent performs preliminary identification of which weight blocks are zero-value and which are dense before the inference process begins. Based on this pre-computed sparsity pattern, the system pre-configures which clusters should be active and which can be powered down, ensuring that only necessary computational resources are consumed while maintaining the ability to assign weights dynamically.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances the efficiency of artificial neural network inference by minimizing trivial computations and reducing energy usage by approximately 10 times compared to traditional methods.
Implementation Method 1
Each of the memristors may be resistive random access memory (RRAM), conductive-bridging random access memory (CBRAM), phase-change memory (PCM)
Implementation Method 2
conductive-bridging random access memory (CBRAM)
Data Source
AI summary
A method of storing a sparse weight matrix for a trained artificial neural network in a circuit including a series of clusters. The method includes partitioning the sparse weight matrix into at least one first sub-block and at least one second sub-block. The first sub-block includes only zero-value weights and the second sub-block includes non-zero value weights. The method also includes assigning the non-zero value weights in the at least one second sub-block to at least one cluster of the series of clusters of the circuit. The circuit is configured to perform matrix-vector-multiplication (MVM) between the non-zero value weights of the at least one second sub-block and an input vector during an inference process utilizing the artificial neural network. The sub-blocks containing all zero elements are power gated, thereby reducing overall energy consumption for inference.