Neural network model optimization method and system based on ThLU function

Abstract

The invention provides a neural network model optimization method and system based on a ThLU function. According to the invention, the positive semi-axis based on the ReLU function has no gradient disappearance phenomenon and the negative semi-axis based on the tanh function can reduce the neuron death phenomenon; pROVIDING A NOVEL ACTIVATION FUNCTION, wherein the unit is a correction linear unitbased on a tanh function. The negative half axis of the tanh function and the positive half axis of the ReLU function are integrated. Based on a CIFAR-10 data set and a CIFAR-100 data set., the tanh function, the ELU function, the LReLU function, the ReLU function and the ThLU function are respectively verified through a VggNet-16 neural network architecture; verification shows that the neural network model obtained on the basis of ThLU function training has better accuracy and lower loss, the neuron death phenomenon of the ReLU function is effectively solved, and the neural network model is amore efficient activation function.

Description

technical field [0001] The present invention relates to the technical field of deep learning, and more specifically, relates to a ThLU function-based neural network model optimization method and system. Background technique [0002] In recent years, deep learning theory has achieved fruitful research results in image recognition, image detection, speech recognition, and lip language recognition, which has improved the level of artificial intelligence. Part of the reason why deep learning theory has achieved remarkable results is: the improvement of deep learning network architecture, the improvement of computer hardware, the improvement of activation functions, and the improvement of optimization algorithms. Among them, the improvement of activation function is an important reason for the remarkable achievements of deep learning theory. The activation function originated from logistic regression. In order to transform the linear net input into a nonlinear equation with good...

AI Technical Summary

Problems solved by technology

[0004] In order to overcome the problem in the prior art that the death of neurons in the neural network function makes it difficult for the neural network to continue deep learning, the present invention provides a method and system for optimizing the neural network model based on the ThLU function, so as to improve the accuracy of the neural network model and improve the accuracy of the neural network model. Reduce loss and improve the deep learning ability of neural network

Images