Multiply-Accumulate Circuit Calibration for Precise Reservoir Weights
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network algorithms, such as RNN and LSTM, require significant computational resources and time for learning due to high operation amounts, and reservoir computing demands high precision in weights between reservoir and output units, making it challenging to achieve high-speed operation with sufficient precision.
Innovation Solution
A multiplication device comprising a short-term memory circuit, a long-term memory circuit, and a conversion circuit, which generates control voltages to adjust conductance for precise weight-based multiplication operations, allowing for high-resolution and high-precision multiplication processes with efficient calibration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If RNN or LSTM is used for time-series data processing with general-purpose processors, then the algorithm can be implemented, but the operation amount increases significantly requiring large amounts of time and power consumption
Solution Approach 1:
The patent segments the neural network processing into two distinct parts: a fixed reservoir computing layer that handles time-series data processing, and a simplified output layer with trainable weights. This segmentation eliminates the need for backpropagation through time in the reservoir layer, dramatically reducing learning time while maintaining algorithm functionality.
Solution Approach 2:
The reservoir layer acts as an intermediary between the input time-series data and the output layer. It transforms the input data into a higher-dimensional space where the subsequent linear classification becomes simpler and faster, reducing the overall computational burden.
2Adaptability or versatility
If RNN or LSTM is used for time-series data processing, then the algorithm can be implemented, but the operation amount increases requiring large amounts of power consumption
Solution Approach 1:
By segmenting the processing into a fixed reservoir layer and a simplified output layer, the patent eliminates the computationally intensive backpropagation through time operation, significantly reducing power consumption while maintaining the ability to process time-series data.
Solution Approach 2:
The patent extracts the complex temporal processing functionality into the reservoir layer with fixed weights, removing the need for iterative gradient computation in that layer. Only the simple output layer requires learning, dramatically reducing overall power consumption.
3Productivity
If reservoir computing is used for time-series data processing, then the operation amount is reduced, but high precision in the weights between reservoir unit and output unit is required
Solution Approach 1:
The patent replaces traditional high-precision analog weight implementations with digital or mixed-signal weight representations. This substitution allows for sufficient precision with lower bit-depth weights, reducing hardware complexity while maintaining the operational efficiency benefits of reservoir computing.
Solution Approach 2:
The patent changes the precision parameter of the weights from high-precision analog values to lower-precision digital representations. This parameter change maintains adequate performance for most applications while significantly simplifying the hardware requirements for weight storage and processing.
4Measurement precision
If high precision in weights is provided, then the desired output signal is achieved, but it is technically difficult to provide an operator that can operate at high speed while maintaining sufficiently high precision
Solution Approach 1:
The patent segments the system into a large fixed-weight reservoir layer that operates at high speed with simple lookups, and a small output layer with fewer trainable parameters. This segmentation allows the majority of processing to occur at high speed without being constrained by high-precision weight update requirements.
Solution Approach 2:
The patent accepts that the reservoir weights are fixed and non-trainable, treating them as disposable pre-computed parameters. This approach allows using simpler, faster hardware for weight storage and retrieval, sacrificing the ability to update these weights but gaining significant speed advantages.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The device enables efficient and precise multiplication operations, reducing computational time and resources while maintaining high precision, suitable for applications in neural networks and reservoir computing.
Implementation Method 1
短时记忆电路(12)configured to hold an electric charge and generate a first control voltage in accordance with the weight value by the held electric charge
Implementation Method 2
The control circuit is configured to execute a calibration process of matching the first control voltage generated by the short-term memory circuit with the second control voltage generated by the long-term memory circuit. The calibration process is executed by transferring an electric charge from the long-term memory circuit to the short-term memory circuit.
Implementation Method 3
The conversion circuit is configured to change a conductance in accordance with a voltage applied to the control terminal and output, from the output terminal, a current obtained by multiplying a voltage applied to the input terminal by the conductance
Data Source
AI summary
A multiplication device according to one embodiment includes a short-term memory circuit, a long-term memory circuit, a conversion circuit, and a control circuit. The short-term memory circuit generates a first control voltage in accordance with a weight value. The long-term memory circuit generates a second control voltage by a circuit with a larger time constant than the short-term memory circuit. The conversion circuit outputs an output current by multiplying an input voltage by a conductance. The output current is output by that, the first control voltage is applied to a control terminal of the conversion circuit, and an input voltage according to an input value is applied to an input terminal of the conversion circuit. The control circuit executes a calibration process of matching the first control voltage with the second control voltage by transferring an electric charge from the long-term memory circuit to the short-term memory circuit.


