Neural Network Parameter Weight Memory Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks face challenges in error resilience and power management due to differences in error sensitivity across layers, where errors in input layers have less impact but are more frequent, while errors in output layers result in significant inaccuracies, necessitating efficient memory allocation for parameter weights.
Innovation Solution
The method involves defining layers in a neural network based on their proximity to the input and output, associating each layer with memory banks of varying error resilience and power consumption, storing parameter weights from less critical layers in error-prone, low-power memory and those from critical layers in more stable, high-power memory.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If parameter weights from all layers are stored in high-resilience memory, then error resilience is improved, but power consumption increases
Solution Approach 1:
The patent applies local quality by differentiating memory allocation based on layer position. Layers closer to the output (with higher error impact) are assigned to high-resilience memory banks, while layers closer to the input (with lower error impact) are assigned to low-resilience memory banks. This localized differentiation optimizes the balance between error resilience and power consumption by applying high reliability only where most needed.
2Use of energy by moving object
If parameter weights from input layers are stored in low-power memory, then power consumption is reduced, but error resilience deteriorates
Solution Approach 1:
The patent changes the parameter of memory resilience based on the parameter of layer position. By evaluating the distance of each layer from the output and transforming this spatial parameter into a memory resilience parameter, the system dynamically assigns appropriate memory types. This parameter transformation allows low-power memory to be used for input layers where error impact is minimal, while maintaining high resilience for output layers.
3Device complexity
If uniform memory allocation is used for all layers, then device complexity is reduced, but error resilience deteriorates
Solution Approach 1:
The patent implements local quality by creating a differentiated memory allocation strategy based on layer position. Instead of uniform allocation, each layer is evaluated based on its distance from the output, and memory banks are assigned with resilience levels matching their criticality. This localized approach maintains manageable complexity through automated evaluation while significantly improving overall error resilience.
Data Source
AI summary
A computer-implemented method optimizes a neural network. One or more processors define layers in a neural network based on neuron locations relative to incoming initial inputs and original outgoing final outputs of the neural network, where a first defined layer is closer to the incoming initial inputs than a second defined layer, and where the second defined layer is closer to the original outgoing final outputs than the first defined layer. The processor(s) define parameter criticalities for parameter weights stored in a memory used by the neural network, and associate defined layers in the neural network with different memory banks based on the parameter criticalities for the parameter weights. The processor(s) store parameter weights used by neurons in the first defined layer in the first memory bank and parameter weights used by neurons in the second defined layer in the second memory bank.