Sparse self-encoder fast training method based on pseudo inverse learning

Abstract

The invention relates to a sparse self-encoder fast training method based on pseudo inverse learning, aiming to overcome the defects of a deep neural network training algorithm in the prior art and to provide a method for sparse self-encoding neural network fast training. The method for sparse self-encoder fast training adopts a pseudo inverse learning algorithm and the biased ReLU activation function. The setting of the number of hidden neurons is directed with the dimension of the input data vector of the self-encoder, the pseudo inverse matrix of the input data of the self-encoder is cut off, the pseudo inverse cut-off matrix is taken as the encoder connection weight, the input data is mapped to the hidden space via the biased ReLU activation function, and the pseudo inverse solution is solved by a pseudo inverse learning algorithm to further obtain the encoder connection weight. The method for sparse self-encoding neural network fast training is free of the iterative optimization based on gradient descending and is free of setting the control parameters, is fast in calculation speed, and can guarantee the learning of the sample sparsity. The method is easy to control reconstruction errors, easy to use, and facilitates hardware realization.

Description

technical field [0001] The invention relates to a fast training method for a sparse autoencoder in the field of artificial intelligence, in particular to a fast training method for a sparse autoencoder based on a pseudo-inverse learning algorithm. Background technique [0002] At present, in the artificial intelligence technology represented by deep learning, supervised learning is usually used, which often requires a large amount of labeled data to train the deep network model. However, most of the data obtained in practical applications are unlabeled. Data, if manual labeling of a large amount of unlabeled data requires high manpower and time costs. Therefore, using unsupervised learning techniques and methods to directly perform representation learning on unlabeled data and make full use of a large amount of unlabeled data is the development trend of artificial intelligence technology. Autoencoder is a commonly used basic model of deep learning. Its basic idea is that th...

AI Technical Summary

Problems solved by technology

Since these algorithms iteratively update the model parameters based on each individual training sample (or a small part of the sample), it often takes multiple iterations to obtain the solution to the optimization problem. When the amount of data is large, the training process is extremely time-consuming

Secondly, the training algorithm involves many control parameters, such as the maximum number of iterations (maximum epoch), learning step length (step length), weight decay factor (weight decay), momentum factor (momentum), these parameters directly affect the final training results, But how to set these control parameters lacks theoretical basis

When these parameters are not selected properly, useful features cannot be learned, and the sparsity of the hidden layer output h cannot be guaranteed.

Images