Neuromorphic Circuit Memristor Synapse Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neuromorphic circuits face challenges in implementing spiking neural networks with high precision and integrability due to the limitations of CMOS and optical technologies, which result in reduced performance and inefficiencies in training, particularly with the von Neumann bottleneck and the difficulty in optimizing overall objective functions using existing techniques.
Innovation Solution
A neuromorphic circuit utilizing an array of memristor-based synapses with bidirectional connections, a training module that includes an estimation unit for time derivative of spike rates, and a controller to modify synapse values based on firing rates, allowing for local training and high integrability, enabling efficient learning and adaptation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If CMOS or optical technologies are used to produce neurons and synapses, then the circuit can be manufactured with standard processes, but each neuron and synapse occupies several tens of micrometers edgewise, limiting the number of neurons and synapses that can be integrated on a chip
Solution Approach 1:
The patent transitions from planar 2D integration to 3D vertical stacking, where multiple layers of neurons and synapses are stacked in the vertical dimension. This allows significantly more computational elements to be packed into the same chip footprint, resolving the contradiction between manufacturability and chip area utilization.
Solution Approach 2:
The patent implements a hierarchical nested structure where synapses are integrated within or alongside neuron circuits in a compact manner. Multiple functional blocks are nested within each other vertically, maximizing the use of available chip real estate while maintaining standard manufacturing compatibility.
2Use of energy by moving object
If spiking neural networks are implemented with bidirectional synapses and local training, then energy consumption is reduced and performance is improved, but the training precision and optimization of overall objective functions are difficult to achieve
Solution Approach 1:
The patent introduces an intermediary mechanism where time-derivative estimation units and controlled interconnections act as mediators between local spike events and global training objectives. This intermediary layer enables precise gradient computation and weight updates while maintaining the energy efficiency of local training, resolving the contradiction between energy consumption and training precision.
3Adaptability or versatility
If von Neumann architecture with spatially separated memory and processor is used, then flexible data processing is enabled, but communication bus congestion occurs between memory and processor during neural network training and inference
Solution Approach 1:
The patent merges memory and processing functions into a unified neuromorphic architecture where synapses (memory elements) are directly integrated with neuron processing circuits. This eliminates the von Neumann bottleneck by combining what were previously separate components, simultaneously achieving both flexibility and high processing speed.
4Productivity
If the number of neurons and synapses is increased to improve neural network performance, then better task solving capability is achieved, but the chip area required increases due to each element occupying several tens of micrometers
Solution Approach 1:
The patent exploits the vertical dimension through multi-layer stacking, enabling thousands of additional neurons and synapses to be integrated without increasing chip footprint. This vertical scaling approach directly resolves the contradiction by providing high-performance capability within limited area constraints.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The proposed solution enables precise training of spiking neural networks with reduced power consumption and increased performance, achieving energy savings of at least two orders of magnitude compared to GPU von Neumann architectures, while maintaining high integrability and enabling efficient learning during use.
Implementation Method 1
A memristor is a passive electronic component. The name is a portmanteau word formed from the two English words memory and resistor. A memristor is a non-volatile memory component, the value of the electrical resistance thereof changing with the application of a voltage for a certain length of time and remaining at said value in the absence of voltage.
Data Source
AI summary
A neuromorphic circuit implementing a spiking neural network and including bidirectional synapses made by a set of memristors arranged in an array, neurons firing spikes at a variable rate and connected to neurons via a synapse, and a neural network training module including, for at least one bidirectional synapse, an estimation unit obtaining an estimation of the time derivative of the spike rate of each neuron, an interconnection having at least two positions between the synapse and each neuron, and a controller sending a control signal to the interconnection after a spike, the signal changing the position of the interconnection, so as to connect the estimation unit and the synapse.