Neural network parameter optimization method and system and electronic equipment

Abstract

The invention relates to a neural network parameter optimization method and system and electronic equipment. The method comprises the steps that a, a neural network is trained, whether the neural network falls into a locally optimal solution is detected in the training process, and if the neural network falls into the locally optimal solution, the step b is executed; b, jump-out operation is executed, wherein a non-normalized gradient calculation method is utilized to break a weight update direction of the neural network, a new update direction is selected, and the weight of the neural networkis updated along the new update direction; and c, re-optimization operation is executed, wherein parameter re-optimization is performed on the neural network through an optimization algorithm, so that the neural network finds a new optimal solution. Through the neural network parameter optimization method and system and the electronic equipment, the non-normalized gradient descent method is utilized to break the original gradient descent update direction, the weight of the neural network is updated along a different direction, continuous iteration is performed with a re-optimization module, and the problem that system performance cannot be improved after the neural network falls into the locally optimal solution is solved.

Description

technical field [0001] The present application relates to the technical field of neural networks, in particular to a method, system and electronic equipment for parameter optimization of neural networks. Background technique [0002] In 2017, China's artificial intelligence ushered in a new era of rapid development. Among them, deep learning has become the most widely used, and has made remarkable achievements in image recognition, Go and other aspects. [0003] In the training of deep neural networks, the gradient descent method has become the most widely used optimization method. This method continuously corrects the parameters of the neural network by calculating the loss function so that the deviation is continuously propagated backwards through the gradient. When the number of neural network layers is very large, when the error is very small, the value propagated backward through each layer of neural network gradually decreases or even disappears. In neurons, the mor...

AI Technical Summary

Problems solved by technology

Although this type of method has a strong search ability, too many particles will definitely consume more computing resources and memory resources

In addition, the number of parameters in the neural network is now as many as one million. The method o

Images