Neural Network Memory Device Weight Update Mechanism
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network apparatuses face challenges in updating weights during the learning process, requiring high precision and speed, especially when weights need to be updated by a minute amount, which is difficult with binary weights used for inference.
Innovation Solution
A memory device with a first and second accumulation circuit, an update circuit, and an output circuit that accumulates electric charges and updates weights by changing the difference between these circuits based on an update amount, allowing for high-precision and high-speed weight updates using continuous values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If binary weights are used for inference, then device complexity is reduced, but manufacturing precision deteriorates because learning requires continuous values with about 1000 precision levels
Solution Approach 1:
The weight update operation is segmented into two independent accumulation circuits: a first accumulation circuit for updating weights in the positive direction and a second accumulation circuit for updating weights in the negative direction. Each circuit independently accumulates update amounts, allowing the system to handle continuous weight values with high precision while using simple binary representation (positive/negative accumulation) for the final weight state.
2Measurement precision
If high-precision weight updates are performed during learning, then learning accuracy is improved, but processing speed deteriorates due to the complexity of handling continuous values
Solution Approach 1:
The patent replaces complex mechanical/computational weight update mechanisms with an electrical accumulation system. Two accumulation circuits use electrical charge accumulation to represent weight values, where the first circuit accumulates positive updates and the second accumulates negative updates. This electrical approach enables both high precision (through fine-grained charge accumulation) and high speed (through parallel electrical operations) simultaneously.
3Productivity
If continuous weight values are updated at high speed, then learning efficiency is improved, but device complexity increases due to the need for precise control circuits
Solution Approach 1:
The patent merges the weight update control functionality into the accumulation circuits themselves. The first and second accumulation circuits are integrated with control logic that automatically manages positive and negative updates based on learning signals. This merging eliminates the need for separate complex control circuits, as the accumulation circuits inherently handle both the precision control and the high-speed update operations through their unified electrical accumulation mechanism.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables efficient and precise weight updates during the learning process, allowing for high-precision learning in neural networks by representing weights as differences in electric charge accumulation, facilitating fast and accurate adjustments.
Implementation Method 1
a first accumulation circuit which accumulates electric charges; a second accumulation circuit which accumulates electric charges
Implementation Method 2
The update circuit is configured to change, in accordance with an update amount for updating the weight, a difference between a charge amount accumulated in the first accumulation circuit and a charge amount accumulated in the second accumulation circuit
Data Source
AI summary
A memory device according to an embodiment can be used for storing weights for a neural network. An update circuit changes a difference between charge amounts accumulated in first/second accumulation circuits in the memory device. An output circuit outputs, as a weight, a signal corresponding to the difference between the charge amounts. The update circuit performs the change of the difference by changing, when the update amount is positive, the electric charges accumulated in the first accumulation circuit in a first direction by a charge amount corresponding to an absolute value of the update amount, the first direction being either an increasing direction or a decreasing direction, and changing, when the update amount is negative, the electric charges accumulated in the second accumulation circuit in the first direction by a charge amount corresponding to an absolute value of the update amount.