Activation function generation method of neural network model

Abstract

The present invention discloses an activation function generation method of a neural network model. The method includes the following steps that: S1, a plurality of different basic activation functions are selected; S2, the different basic activation functions selected in the S1 are combined so as to form the activation function of the neural network model; and S3, the activation function of the neural network model is updated with the iteration of the neural network model through a back propagation means. With the activation function generated by using the method of the invention can improve the nonlinear change learning capability of the neural network model, and the neural network model can respond differently to different input in a test stage, and a defect that a neural network uses a single activation function can be eliminated.

Description

technical field [0001] The invention relates to a field related to machine learning, in particular to a method for generating an activation function of a neural network model. Background technique [0002] In recent years, deep learning has achieved remarkable results in the field of computer vision, and one of the important factors is the development of activation functions. In the artificial neural network, the activation function of the neuron node defines the mapping to the output of the neuron. Simply put, the output of the neuron is processed by the activation function and then used as the output. [0003] The main function of the activation function in the neural network is to provide the nonlinear modeling ability of the network. Assuming that an example neural network only contains linear convolution and fully connected operations, then the network can only express linear mapping, even if the depth of the network is increased, it is still a linear mapping, and it i...

AI Technical Summary

Problems solved by technology

[0005] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and provide a method for generating an activation function of a neural network model. The activation function generated by the method improves the ability of the neural network model to learn nonlinear changes, and makes the neural network model In the test phase, it can respond differently to different inputs, which improves the shortcomings of the neural network using a single type of activation function.

Images