Incremental learning method based on Fisher information matrix

Abstract

The invention discloses an incremental learning method based on a Fisher information matrix. According to the method, the importance of parameters of each layer of the neural network is judged by calculating a Fisher information matrix; by separating interlayer parameters and intra-layer parameters, the calculation efficiency of the Fisher information matrix is greatly improved; by introducing new vehicle incremental data and iteratively training the model, the huge calculation amount of full data training is avoided; corresponding weight coefficients are set for parameters with different importance, so that the priori model and the posteriori model have the maximum similarity; the Fisher information matrix is associated with the classical KL divergence, and more powerful support is provided for manifold hypothesis of the neural network. The test result shows that parameters of all layers of the neural network indeed have different importance, and the distance between the prior model and the posterior model of the neural network can be effectively controlled by introducing the regular term, so that the distance is as small as possible.

Description

technical field [0001] The invention belongs to the technical field of computers, in particular to an incremental learning method based on Fisher information matrix. Background technique [0002] The traditional incremental learning method is to inherit the old parameters as the initial value of the model. This operation is indeed effective, but the improved accuracy is only half of that of joint training. Because the process of neural network learning is the update of parameters, the update of parameters will force the model to forget the old data. The experimental results also clearly show that using a high learning rate makes the new model perform worse on the old data, although it can perform better on the current training set. Contents of the invention [0003] The purpose of the present invention is to provide an incremental learning method based on Fisher information matrix aiming at the deficiencies of the prior art. [0004] The purpose of the present invention ...

AI Technical Summary

Problems solved by technology

The experimental results also clearly show that using a high learning rate makes the new ...

Images