A neural network-optimized image recognition method

Abstract

The invention discloses a neural network optimization method, called as a neural network optimization method of a lifted proximal operator machine (LPOM), and relates to the technical field of deep learning. The neural network optimization method disclosed by the invention has the advantages that existing first-order or second-order derivative information does not need to be used for directly optimizing a neural network, instead, a neural network is converted into a new LPOM optimization problem, and then the LPOM optimization problem is solved by virtue of an alternate minimization method. Byadopting the method disclosed by the invention, a layered structure of the neural network is eliminated in a solving process; solving can be carried out by virtue of an alternant iteration process; asolving method can be converted into a random algorithm in some degree, so that smaller calculation amount is maintained during iteration; and for a neural network more than 20 layers, training errorstill can be stably reduced by adopting the method disclosed by the invention.

Description

technical field [0001] The present invention relates to the field of deep learning technology, in particular to a new neural network optimization method named Lifted Proximal Operator Machine (LPOM). This method transforms the optimization problem of the neural network into a new optimization problem for solution, and the Karush–Kuhn–Tucker (KKT) condition of the converted optimization problem is equivalent to the forward process of the neural network. Background technique [0002] In recent years, deep neural networks have achieved great success in the fields of artificial intelligence, image recognition, and speech recognition. Compared with shallow neural networks, deep neural networks often have more model parameters and larger capacity, and can achieve better results when the amount of data is large. However, solving the optimization problem of a neural network is a typical non-convex optimization problem. As the number of neural network layers increases, it becomes mo...

AI Technical Summary

Problems solved by technology

However, solving the optimization problem of the neural network is a typical non-convex optimization problem. As the number of neural network layers increases, it becomes more difficult to train the neural network.

However, in practice, ADMMs often only solve shallow neural networks (about 4 layers)

For deep neural networks, the convergence speed of the ADMM method will be very slow

Images