Nonzero Convolution Circuit for Neural Network Memory Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks require a large amount of memory and computing units, making it difficult to achieve high-performance implementation due to physical limitations in memory access, especially with the increasing number of neurons, which necessitates a reduction in memory usage and on-chip implementation.
Innovation Solution
A neural network circuit device with a nonzero convolution operation circuit that skips zero weights and performs operations only on nonzero weights, combined with a sum circuit and an activating function circuit, allowing for reduced memory usage and on-chip implementation by optimizing convolution operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a deep neural network with increased number of neurons is used to improve recognition accuracy and functions, then the recognition accuracy and network capabilities are improved, but the memory requirements and computing unit requirements increase significantly
Solution Approach 1:
The patent extracts and removes zero-weight connections from the neural network model. By identifying and eliminating weights that are exactly zero, the system reduces the number of parameters that need to be stored in memory without affecting the network's computational output or recognition accuracy. This extraction principle directly addresses the contradiction by reducing memory requirements while preserving network functionality.
Solution Approach 2:
The patent segments the weight matrix into nonzero elements and zero elements. By separating the weight representation into these two distinct segments, the system can store only the nonzero weights in memory, significantly reducing the memory footprint. The segmentation allows the network to maintain its full computational capability while using less memory to store only the essential weight parameters.
2Measurement precision
If a deep neural network with increased number of neurons is used to improve recognition accuracy, then the network capabilities are improved, but the number of computing units and computational complexity increase
Solution Approach 1:
The patent extracts zero-weight operations from the computational process. By identifying and removing multiplications by zero from the forward and backward propagation algorithms, the system reduces the actual number of computational operations that need to be performed. This extraction reduces computational complexity while maintaining the same recognition accuracy, as zero-weight connections contribute nothing to the computational output.
Solution Approach 2:
The patent segments the computational process into operations involving nonzero weights and operations involving zero weights. By separating these operations, the system can eliminate the unnecessary zero-weight operations from the computational pipeline, reducing the number of multiply-accumulate operations and simplifying the computational graph without affecting the network's ability to achieve high recognition accuracy.
3Productivity
If zero coefficient skipping is applied to reduce computation and memory bandwidth, then the computational efficiency is improved, but the device complexity increases due to sparse matrix operations
Solution Approach 1:
The patent merges the sparse weight storage format with the computational engine design. By integrating the sparse matrix representation directly into the neural network hardware architecture, the system achieves efficient zero-skipping operations without requiring separate complex sparse matrix handling logic. This merging reduces device complexity compared to implementing sparse operations as a separate layer, while still achieving high computational efficiency through zero-coefficient skipping.
Data Source
AI summary
A tri-state neural network circuit includes, in an intermediate layer: an input value Xi to which convolution is applied; a nonzero convolution operation circuit configured to receive a weight Wi and to perform a convolution operation thereof; a sum circuit configured to take a sum of operation values each subjected to the convolution operation and a bias W0; and an activating function circuit configured to convert a signal Y generated by taking the sum, using an activating function f(u). The nonzero convolution operation circuit is configured to skip a weight Wi having a zero weight and to perform a convolution operation based on a nonzero weight and an input value Xi corresponding thereto.