Convergence method and device for a high-dimensional deep learning model

Abstract

The embodiment of the invention discloses a convergence method and device for a high-dimensional deep learning model. The method includes performing a unit iteration on the model according to the first position of the error surface to determine a second position of the solution of the model on the error surface; according to a unit iteration, determining the gradient and curvature of the second position relative to the error surface, And determine the precision growth efficiency and model error of the solution of the model according to the first position and the second position; According to the gradient, curvature, precision growth efficiency and model error, determine whether the second position is a saddle point or a high noise point on the error surface; When the second position of the error surface is a saddle point or a high noise point, the batch number of the next unit iteration is adjusted. In the embodiment of the present invention, the batch number of the model in the next batch of iterations is determined according to the saddle point or high noise. In this way, by adjusting the gradient and random estimation noise in the iterative process, the accuracy growth efficiency of the solution of the model can be optimized and the saddle point can be escaped.

Description

technical field [0001] The invention relates to the technical field, in particular to a convergence method and device for a high-dimensional deep learning model. Background technique [0002] With the vigorous development of big data in all walks of life, many applications in the field of artificial intelligence appear in our lives through deep learning methods. Deep learning mimics the workings of the human brain by building deep neural networks. This deep neural network mechanism has made breakthroughs in speech recognition, image recognition, natural language processing and other fields in recent years. [0003] The parameters of the deep neural network are very large, and can reach the level of tens of millions or even hundreds of millions. For deep learning model training, including the use of mini-batch stochastic gradient descent (MBGD, Mini-batch Gradient Descent) method to seek the optimal solution of the model. [0004] The advantages of MBGD are mainly fast tra...

AI Technical Summary

Problems solved by technology

[0007] In large-scale machine learning problems, the method of optimizing the accuracy growth efficiency of the model's solution and escaping the saddle point by adjusting the gradient and the amount of random estimation noise in the iterative process is relatively blank.

Images