Neural Network Weight Encoding Circuit for Memory Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks require large amounts of memory and processing power due to the large number of weights between layers, which can lead to increased power consumption and reduced execution speed.
Innovation Solution
A neural network circuit that includes a weight memory for storing encoded weights, where the encoded weights consist of an index weight word and a decompression logic circuit that decodes these weights to obtain a sequence of non-pruned and pruned weight words, which are then provided to input-weight multipliers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all weight values are stored in memory, then complete neural network functionality is maintained, but memory requirements and processing complexity increase significantly
Solution Approach 1:
The patent extracts and removes pruned weight words (zero-value weights) from the stored weight data. By storing only non-pruned weight words along with pruning metadata (bitmaps indicating which weights are zero), the system reduces memory requirements while maintaining the ability to reconstruct complete weight sequences for neural network operations
Solution Approach 2:
The patent changes the representation parameters of weight data by introducing a compressed format that includes pruning metadata. Instead of storing all weight values uniformly, the system uses differential encoding where pruned weights are represented by metadata bits rather than full weight values, reducing the quantity of stored data while preserving functional equivalence
2Measurement precision
If all weight values are stored and processed, then accurate neural network computation is achieved, but processing time and power consumption increase
Solution Approach 1:
The patent performs preliminary action by pre-computing and storing pruning metadata (bitmaps) that identify which weight words are zero. During inference, this metadata allows the system to skip processing of pruned weights entirely, reconstructing only non-zero weight sequences and thereby reducing processing time and power consumption while maintaining computational accuracy
3Quantity of substance
If weight pruning is applied to reduce memory usage, then memory requirements decrease, but decoding complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the weight data into non-pruned weight words and pruning metadata (bitmaps). This segmentation allows the system to store weight data in a compressed format where only essential information (non-zero weights and their positions) is retained, reducing memory usage while enabling efficient decoding through simple bitmap-based reconstruction logic
Data Source
AI summary
According to an aspect, a neural network circuit for decoding weights of a neural network includes a weight memory configured to store encoded weights for the neural network, where the encoded weights includes an index weight word, and a decompression logic circuit configured to retrieve the encoded weights from the weight memory, decode the encoded weights using the index weight word to obtain a sequence of one or more non-pruned weight words and one or more pruned weight words, and provide the sequence of the non-pruned weight words and the pruned weight words to a plurality of input-weight multipliers.