Convergence method of high-dimensional depth learning model and device

Abstract

The embodiment of the invention discloses a convergence method of a high-dimensional depth learning model and a device. The method includes the steps of performing one unit iteration on a model according to a first position of the error surface to determine a solution of the model at a second position of the error surface; determining a gradient and a curvature of the second position relative to the error surface according to one unit iteration, and determining an accuracy increasing efficiency and a model error of the solution of the model according to the first position and the second position; determining whether the second position is a saddle point or a high noise point of the error surface based on the gradient, the curvature, the accuracy increasing efficiency and the model error; and when the second position of the error surface is a saddle point or a high noise point, adjusting the batch number of the next unit iteration. Embodiments of the present invention determine the batch number of the model at the next batch iteration based on the saddle point or high noise. Therefore, it is possible to optimize the accuracy increasing efficiency and the escape saddle point of the model by adjusting the gradient and stochastic noise estimation in the iteration process.

Description

Technical field [0001] The present invention relates to the technical field, in particular to a method and device for convergence of a high-dimensional deep learning model. Background technique [0002] With the vigorous development of big data in all walks of life, many applications in the field of artificial intelligence appear in our lives through deep learning methods. Deep learning simulates the working principle of the human brain by constructing deep neural networks. This kind of deep neural network mechanism has made breakthrough progress in speech recognition, image recognition, natural language processing and other fields in recent years. [0003] The amount of parameters of deep neural networks is very large, which can reach tens of millions to hundreds of millions. For deep learning model training, including the use of small batch stochastic gradient descent (MBGD, Mini-batch Gradient Descent) method, so as to find the optimal solution of the model. [0004] The advant...

AI Technical Summary

Problems solved by technology

[0007] In large-scale machine learning problems, the method of optimizing the accuracy growth efficiency of the model's solution and escaping the saddle point by adjusting the gradient and the amount of random estimation noise in the iterative process is relatively blank.

Images