Neural Network Weight Calibration Using Non-Volatile Memory Synapses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial neural networks face challenges in achieving high-performance information processing due to a lack of adequate hardware technology, particularly in terms of energy efficiency and scalability, as they rely on bulky CMOS-implemented synapses and digital supercomputers.
Innovation Solution
Utilizing non-volatile memory arrays, such as split gate flash memory cells, to implement synapses that can be individually programmed, erased, and read without affecting other memory cells, enabling continuous analog programming for precise tuning of synapse weights, thereby reducing the need for separate multiplication and addition logic circuits.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If CMOS analog circuits are used to implement synapses, then the neural network can be implemented with standard technology, but the hardware becomes bulky and energy efficiency deteriorates
Solution Approach 1:
The patent merges the synapse weight storage function with the multiplication operation by using the conductance state of memory cells to represent weights. This eliminates the need for separate storage and computation hardware, reducing both area and energy consumption while maintaining manufacturability with standard memory technologies.
Solution Approach 2:
The patent replaces bulky CMOS analog circuitry with memory-based implementations. By using memory cells whose conductance states represent synaptic weights, the system eliminates complex analog multiplication circuits while achieving the same computational function with significantly reduced area and improved energy efficiency.
2Productivity
If digital supercomputers or specialized GPU clusters are used, then high computational performance is achieved, but cost increases and energy efficiency deteriorates
Solution Approach 1:
The patent substitutes digital computation with analog computation using memory cell conductances. The Ohm's law-based current flow through memory cells naturally performs multiplication and accumulation operations, eliminating the need for energy-intensive digital processors while maintaining computational performance.
Solution Approach 2:
The patent makes memory cells perform multiple functions: storing synaptic weights, performing multiplication through conductance modulation, and accumulating results through current summation. This multi-functionality eliminates the need for separate computational units, reducing overall system energy consumption while maintaining high throughput.
3Device complexity
If non-volatile memory arrays are used to implement synapses, then energy efficiency and hardware complexity are reduced, but calibration of electrical parameters becomes necessary to compensate for variations
Solution Approach 1:
The patent performs preliminary calibration of memory cell electrical parameters during manufacturing or initialization. By pre-characterizing and storing calibration data, the system compensates for device variations before actual neural network operation, eliminating the need for complex real-time calibration circuits while maintaining computational accuracy.
Solution Approach 2:
The patent creates a digital copy or model of the memory cell electrical characteristics and uses this information to compensate for variations during computation. By measuring and storing calibration parameters, the system can correct for device-to-device variations without adding complex hardware, balancing simplicity with accuracy.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances energy efficiency and reduces hardware complexity by integrating memory cells that can store multiple discrete values, allowing for fine-tuning of neural network weights, thus improving computational performance and reducing power consumption.
Implementation Method 1
Each of the plurality of memory cells store a weight value corresponding to a number of electrons on the floating gate
Implementation Method 2
The plurality of memory cells multiply the first plurality of inputs by the stored weight values to generate the first plurality of outputs
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Numerous examples are disclosed for performing calibration of various electrical parameters in a deep learning artificial neural network. In one example, a system comprises a digital-to-analog converter for receiving an input of k bits and generating a first analog output, a mapping scalar for converting the first analog output into a second analog output, and an analog-to-digital converter for generating an output of n bits from the second analog output, where n is a different value than k.