Parameter regulation method and device for neuron activation function

Abstract

The invention discloses a parameter regulation method and device for a Noisy Softplug neuron activation function. The method comprises the following steps that: setting a parameter to be set as an initial value, and using an activation function which finishes parameter setting for an artificial neural network so as to obtain the highest pattern recognition accuracy rA of the artificial neural network; in a preset LIF (Leaky Integrate-and-Fire) neuron, selecting a noise value [Sigma]0, adopting a least square method to carry out fitting on a corresponding relationship between constant current with different sizes and [Sigma]0 and a pulse spike rate, and determining the value of each parameter to be set; on the basis of the above determined value, updating an initial value, and regulating the activation function; on the basis of the regulated activation function, training the artificial neural network, applying a weight obtained by training to a pulse neural network to obtain the highestpattern recognition accuracy rB of the pulse neural network; and if rA-rB is smaller than a set value, finishing regulating parameters, and otherwise, reselecting [Sigma]0. By use of the method, theartificial neural network can be guaranteed to be trained to obtain the weight suitable for the pulse neural network, and therefore, the pattern recognition accuracy of the pulse neural network is improved.

Description

technical field [0001] The invention relates to the field, in particular to a Noisy Softplus neuron activation function parameter adjustment method and a device thereof. Background technique [0002] With the development of neural networks, deep neural networks have achieved extensive success in computer vision, natural language processing and other fields. The currently widely used ANNs are second-generation neural networks, which have the problems of high energy consumption and low real-time performance, and the new generation of SNNs can solve the above two problems well. However, since the learning algorithm research of deep SNNs is still in its infancy, and the network information transmission in SNNs is based on pulses, the mainstream learning algorithms of deep ANNs are not applicable, so deep SNNs have the problem of difficult training. [0003] Therefore, a training method based on pulse frequency coding was proposed later, using the value in ANNs to represent the ...

AI Technical Summary

Problems solved by technology

[0004] In order to achieve the above goals, it is necessary to ensure that the activation function Noisy Softplus used to train deep ANNs matches the response of biological neurons in deep SNNs, which requires precise setting of the parameters of the activation function NoisySoftplus according to the LIF neuron model of SNNs, However, the current parameter adjustment for Noisy Softplus neuron activation function NoisySoftplus still lacks a specific method

Images