Neuron Circuit Using Threshold-Adjustable MOS Transistor for ReLU Activation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional CMOS circuits implementing ReLU activation functions in neural networks face high power consumption and bulkiness due to frequent memory access and reliance on general-purpose processors for activation function computation, hindering the development of low-power and compact neuron circuits.
Innovation Solution
A low-power and compact neuron circuit design utilizing a MOS transistor with a threshold voltage-adjustable property, where a single neuron transistor acts as a neuron, connected to synaptic arrays, and the threshold voltage is adjusted to implement the ReLU activation function, achieved through ferroelectric polarization or channel doping, enabling efficient computation and reduced power usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional CMOS circuits use four transistors per ReLU neuron cell with frequent memory access, then the neural network computation can be performed, but power consumption increases and circuit size becomes large
Solution Approach 1:
The patent merges the ReLU activation function implementation directly into the synaptic array structure by sharing transistor resources. The same transistors used for weight storage are also used for ReLU computation, eliminating the need for separate neuron cells and reducing memory access requirements.
Solution Approach 2:
The synaptic array transistors are designed to serve multiple functions: weight storage during computation phases and ReLU activation function implementation during other phases. This multi-functionality reduces the overall circuit size and power consumption by eliminating dedicated structures for each function.
2Productivity
If conventional CMOS circuits implement ReLU with separate neuron cells, then the activation function can be computed, but the circuit size becomes large and integration density decreases
Solution Approach 1:
The patent combines the ReLU activation function circuitry with the synaptic array structure, sharing physical resources such as transistors and interconnects. This merging eliminates redundant circuit elements and reduces the overall area required for implementing both weight storage and activation functions.
Solution Approach 2:
The synaptic array components are designed to perform both weight storage and ReLU activation computations. The same physical structures are reused for different computational purposes, maximizing area utilization and reducing the total circuit footprint.
3Adaptability or versatility
If general-purpose processors are used for activation function computation in memory arrays, then flexibility is maintained, but power consumption and processing delay increase
Solution Approach 1:
The patent segments the computational tasks by dedicating specific circuit components within the memory array to handle ReLU activation functions. This segmentation allows parallel processing of activation functions alongside weight storage operations, reducing overall processing delay while maintaining flexibility through configurable transistor operations.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The design significantly reduces energy consumption, delays, and space requirements while maintaining a simple circuit structure, achieving efficient computation and output for ReLU activation functions, suitable for high-density integration and miniaturization.
Implementation Method 1
the threshold voltage-adjustable property of the MOS transistor is achieved through ferroelectric polarization reversal of a gate electrode of a ferroelectric-polarized MOS transistor
Data Source
AI summary
Disclosed is a low-power and compact neuron circuit implementing a ReLU activation function including a first-layer synaptic array, a neuron transistor, a resistor, and a second-layer synaptic array. The neuron transistor is a MOS transistor having a threshold voltage-adjustable property, a gate electrode of the neuron transistor is connected to each voltage output end of the first-layer synaptic array, and a drain electrode of the neuron transistor is connected to each voltage input end of the second-layer synaptic array. Thus, it is possible to satisfy the decision computation and output of different synaptic array output values by adjusting the magnitude of the threshold voltage of the transistor. The neuron circuit requires only one transistor in cooperative connection with the first-layer synaptic array and the second-layer synaptic array to implement the ReLU activation function; therefore, a significant improvement is achieved in terms of energy efficiency, delay reduction, and space utilization.