Deep neural network image recognition method based on structured natural gradient optimization

Abstract

The invention provides a deep neural network image recognition method based on structured natural gradient optimization, and belongs to the field of image recognition based on machine learning and neural network technologies. According to the method, a structured natural gradient descent (SNGD) method is provided, a normalization layer is added, a deep network mode of restructuring image recognition is adopted, the correlation calculation of a global Fisher matrix is decomposed, and finally, the decomposition is converted into optimization by using traditional GD, so that the NGD effect can be achieved. Meanwhile, the invention provides a new local Fisher layer and an implementation scheme thereof. Second-order information is introduced into the local Fisher layer, different attributes of parameters in an image recognition network at different positions are considered, constraints are added to recognition model parameter transformation, and gradient updating can be conducted stably and rapidly. By adopting the method, the image recognition network training has higher convergence speed, and the trained model also has better recognition performance.

Description

technical field [0001] The invention relates to an image recognition method, in particular to a deep neural network image recognition method based on structured natural gradient optimization, and belongs to the field of image recognition based on machine learning and neural network technology. Background technique [0002] In the field of image recognition, how to train the deep neural network model stably and quickly is one of the main challenges faced in the application of deep network-based image recognition. At present, although the commonly used first-order traditional gradient descent (GD) optimization method is simple and effective, it often encounters the problem of ill-conditioned curvature (optimizing the canyon area on the surface). At this time, GD keeps bouncing along the ridge of the canyon, making it difficult to pass through Canyons, resulting in slower convergence during training. [0003] Natural Gradient Descent (NGD for short) is a more powerful optimiza...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to solve the technical problem that natural gradient descent is difficult to apply in the image recognition model based on deep neural network in image recognition, and creatively propose a deep neural network image recognition method based on structured natural gradient optimization

Images