Random weight network generalization ability improvement method and device, and computer readable storage medium

Abstract

The invention discloses a random weight network generalization ability improvement method and device, and a computer readable storage medium. The random weight network generalization ability improvement method disclosed by the invention has the benefits that firstly, an initial output layer weight of a weak random weight network is analytically calculated on a pseudo-residual data set; then an objective function considering the loss and the complexity of a current integrated learning model is designed, and the optimization criterion of an optimal output layer weight is obtained by minimizing the objective function; finally, the optimal output layer weight of the weak random weight network is calculated by taking the initial output layer weight as a heuristic method and in combination withthe derived optimization criterion; the process can be regarded as the re-optimization of the initial output layer weight of the weak random weight network; optimization rules are obtained through theobjective function, and the benefit of re-optimizing the initial output layer weight of the weak random weight network is mainly reflected in that the integrated learning model with a simple structure can get better generalization performance, better over-fitting control ability and smaller prediction variance.

Description

technical field [0001] The invention relates to the field of machine learning, in particular to a method, device and computer-readable storage medium for improving the generalization ability of a random weight network. Background technique [0002] Random Weight Network (RWN) is a full-link feed-forward neural network that does not rely on iterative weight update strategies. Unlike traditional weight update strategies based on error backpropagation methods, RWN randomly selects input layer weights. By solving the pseudo-inverse of the output matrix of the hidden layer, the analytical solution of the weight of the output layer is calculated. Since iterative weight adjustment is avoided, RWN obtains extremely fast training speed. At the same time, the universal approximation theorem theoretically guarantees the convergence of RWN. At present, a lot of research work on improving the generalization ability of RWN has been proposed, mainly focusing on the improvement of the RWN...

AI Technical Summary

Problems solved by technology

[0005] The RWN integrated learning method based on the Boosting mechanism considers both horizontal integration and vertical integration, but the weighting mechanism for misclassified samples is an important factor affecting the generalization ability of the integrated learning model, and the determination of sample weights will increase the training complexity of the integrated learning model

At the same time, the weight of the base learner usually depends on the training accuracy of the base learner. A high-precision base learner is easier to obtain a larger weight, which increases the risk of overfitting of the integrated learning model.

Images