Neural Network Weight Scaling for Layer-Specific Memory Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks face challenges in efficiently scaling weights stored in non-volatile memory cells due to variations in weight distributions across different layers and networks, leading to inefficiencies in current-to-voltage and analog-to-digital conversion processes.
Innovation Solution
A system and method for scaling weights in non-volatile memory cells by using a control gate bias generator to apply a bias voltage based on the neural network's layer and distribution of weights, allowing for more efficient use of input and output circuits.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If weights are stored directly in non-volatile memory cells without scaling, then the full range of weight values can be represented, but the range of values required for current-to-voltage and analog-to-digital conversion becomes excessively large, reducing resolution and performance
Solution Approach 1:
The patent applies parameter changes by introducing a scaling factor that transforms the original weight values into scaled weight values. This changes the parameter range of the weights to match the optimal input range for the conversion circuits, thereby improving measurement precision while maintaining the ability to represent the full spectrum of weight distributions through appropriate scaling factor selection for each layer.
Solution Approach 2:
The patent implements dynamics by making the scaling factor adjustable and layer-specific. Different scaling factors can be applied to different layers of the neural network based on their specific weight distributions, allowing the system to adapt dynamically to varying weight ranges and optimize conversion performance for each layer independently.
2Productivity
If different scaling factors are applied to different layers to optimize conversion, then conversion efficiency improves, but the complexity of weight programming and control increases
Solution Approach 1:
The patent applies preliminary action by pre-calculating and storing the optimal scaling factors for each layer before the neural network operation begins. These scaling factors are determined based on the weight distributions of each layer and are saved in the control circuitry, eliminating the need for real-time calculation during operation and reducing programming complexity while maintaining high conversion efficiency.
Solution Approach 2:
The patent introduces an intermediary scaling mechanism that acts as a mediator between the weight storage memory cells and the conversion circuits. The scaling factor serves as an intermediate parameter that simplifies the interface between these components, allowing each to operate in its optimal range without requiring complex direct integration.
3Measurement precision
If weight values are scaled down to fit conversion circuit ranges, then conversion performance improves, but the dynamic range of representable weights is reduced
Solution Approach 1:
The patent resolves this contradiction through dynamics by implementing layer-specific scaling factors that are optimized for each layer's weight distribution. This allows the system to maintain high conversion accuracy for each layer's specific range while preserving the overall dynamic range across the entire neural network, as each layer operates independently with its own optimized scaling.
Solution Approach 2:
The patent applies local quality by tailoring the scaling factor to the specific characteristics of each layer's weight distribution. Rather than using a uniform scaling factor across all layers, each layer receives a locally optimized scaling factor that matches its specific weight range and distribution characteristics, thereby maintaining both conversion accuracy and weight dynamic range.
Data Source
AI summary
In one example, a system comprises an array of non-volatile memory cells arranged in rows and columns; a control gate bias generator to generate a bias voltage to apply to a control gate line coupled to a row of non-volatile memory cells in the array; and an algorithm controller to configure the control gate bias generator based on the layer of a neural network to be stored in the array.