Over-fitting solution based on low-dimensional manifold regularized neural network

Abstract

The invention discloses an over-fitting solution based on a low-dimensional manifold regularized neural network. Target model definition, over-fitting solution framework, model parametric solution andmodel parameter update are involved in the method. The method comprises the steps that restrictive definition is conducted on a target model, wherein the definition comprises a data set, the label ofthe data set and an average loss function; a framework for solving an over-fitting phenomenon is proposed, network parameters are solved by using a method based on regularization and weight lightening under restrictive conditions, and study capability and robustness are enhanced by proposing a bidirectional noise variable; the training optimum solution is finally obtained by using methods based on counterpropagation and point integration respectively to update a network weight and a coordination function according to the obtained network parameter set. By means of the method, solutions aimingat non solution, locally optimal solution and over-fitting solution of training results of a deep neural network can be provided, and demands on calculation resources are reduced by using an appropriate method to improve the efficiency of actual application calculations.

Description

technical field [0001] The invention relates to the field of neural network calculations, in particular to an over-fitting solution based on a low-dimensional manifold regularized neural network. Background technique [0002] The emergence of deep neural networks has greatly accelerated the speed of change in the field of artificial intelligence. As a brand-new field that has developed rapidly for more than ten years, deep learning has attracted the attention of more and more researchers. It has obvious advantages over shallow models in feature extraction and modeling. Deep learning is good at mining more and more abstract feature representations from raw input data, and these representations have good generalization ability. The application of deep neural networks is very basic, and it overcomes some problems that were considered intractable in artificial intelligence in the past. And with the significant increase in the number of training data sets and the sharp increase...

AI Technical Summary

Problems solved by technology

[0004] Aiming at solving the problem of over-fitting phenomenon in neural network, the purpose of the present invention is to provide a kind of over-fitting solution based on low-dimensional manifold regularization neural network. First, restrictively define the target model, including data set and its Label, average loss function; then propose a framework to solve the overfitting phenomenon, use regularization and lightweight methods to solve network parameters under restrictive conditions, and propose a two-way noise variable to enhance learning ability and robustness; according to The obtained network parameter set is used to update the network weights and coordination functions based on backpropagation and point integration methods, and finally the optimal solution for training is obtained.

Images