CMOS-based neuron activation function circuit and neuron circuit

Abstract

The invention discloses a CMOS-based neuron activation function circuit and a neuron circuit. The neuron activation function circuit comprises an input end; an output end; an input unit which is connected with the input end; the first function unit is connected with the input unit and the output end, and is configured to transmit the current of the input end to the output end according to a ratioof 1: 1 when the input end is a positive signal; and a second function unit which is connected with the input unit and the output end, and is configured to transmit the current 1: k of the input end to the output end when the input end is a negative signal, and k is a parameter greater than zero and less than 1. According to the CMOS-based neuron activation function circuit and the neuron circuit,all technical advantages of a Relu function are reserved, the problems of mean shift, gradient disappearance and neuron death are effectively solved, and the CMOS-based neuron activation function circuit and the neuron circuit have the technical characteristics of simple structure, high precision and large input signal range.

Description

technical field [0001] The invention relates to a neuron activation function circuit based on CMOS, and also relates to a neuron circuit using the activation function circuit. Background technique [0002] Neuron circuits are widely used in artificial neural network circuits and various circuits based on artificial neural network technology. Neuron circuits are the most basic units that make up artificial neural network circuits. The main function is to add all the signals transmitted from the upper level And passed to the next level through a specific activation function. As the most basic unit of artificial neural network circuit, the signal superposition accuracy, scalability and activation function accuracy of neuron circuit are very important. Both digital circuits and analog circuits can realize neuron circuits. With the deepening of artificial neural network research, the shortcomings of traditional digital circuits to realize neural networks are becoming more and mo...

AI Technical Summary

Problems solved by technology

[0005] It has many technical advantages. For example, when x > 0, the gradient is always 1, no gradient dissipation problem, fast convergence speed, small amount of calculation, and less training time; however, because the mean value of the function is non-zero, there is a mean shift; At the same time, in the forward conduction, when x<0, its gradient is 0, causing the neuron to fail to activate

Images