Small sample online training method and device and storage medium

Abstract

The invention discloses a small sample online training method and device and a storage medium, and belongs to the technical field of calculation, reckoning or counting. The method comprises the following steps: training a neural network through a data set to obtain a weight parameter, and taking the weight parameter as an original training parameter of a pre-training model; collecting and marking a specified preset quantity of samples; and inputting the marked samples and the original training parameters into a pre-training model for training so as to update the original training parameters. The pre-training model is adopted to train and update original training parameters, the problems that in the prior art, an optimization algorithm is limited by influences of storage, power consumption and computing power and is difficult to be carried on mobile equipment and an embedded platform for use, and a small number of samples are prone to causing overfitting are solved, and meanwhile, in the process of online training of new samples, the optimization efficiency is greatly improved. And disastrous forgetting of old samples can be caused.

Description

technical field [0001] The invention relates to a small sample online training method, device and storage medium, belonging to the technical field of calculation, calculation or counting. Background technique [0002] In recent years, deep neural networks have achieved unprecedented success in speech recognition, image recognition and medical applications, among which convolutional neural networks have a wide range of applications in the field of visual recognition, such as handwritten digit recognition, character recognition, face detection, Cross-media search, face recognition, facial expression analysis, car detection. For the parameter optimization process of neural network training, different learning algorithms have been proposed to train neural networks, among which the gradient-based algorithm, that is, the gradient descent algorithm, is the most used. However, realizing online training of deep neural networks on hardware has always been a difficult problem for rese...

AI Technical Summary

Problems solved by technology

[0003] First of all, the current mainstream optimization algorithms have complex calculations and a large number of training parameters, which are limited by storage, power consumption and computing power, and are difficult to carry on mobile devices and embedded platforms.

Secondly, in practical application scenarios, it is often difficult to collect more samples, and too little data will have a negative impact on model training. For example, because a small amount of training data is difficult to provide enough information for the model to learn, the model may not be able to converge ;The model learns completely irrelevant information of the training data, which makes the model overfit the training data; it is easy to fall into the local optimal value, etc.

Again, in the process of continuous online training of new samples, it will cause catastrophic forgetting of old samples

Images