A meta-learning algorithm based on stepwise gradient correction of a meta-learner

Abstract

The invention discloses a meta-learning algorithm based on stepwise gradient correction of a meta-learner, and the algorithm comprises the steps: firstly, obtaining training data with noise marks anda small amount of clean unbiased metadata sets; establishing a meta-learner, namely a teacher network, on the metadata set relative to a classifier, namely a student network established on the training data set; and carrying out united updating of student network parameters and teacher network parameters by using random gradient descent; obtaining a student network parameter gradient update function through a student network gradient descent format; feeding the network parameters back to the teacher network, and updating the teacher network parameters by using metadata to obtain a corrected student network parameter gradient format; and then updating the student network parameters by using the correction format. Accordingly, the student network parameters can achieve better learning in thecorrection direction, and the over-fitting problem of noise marks is weakened. The method has the characteristics of easiness in understanding, realization, interpretability and the like of a user, and can be robustly suitable for an actual data scene containing noise marks.

Description

technical field [0001] The invention belongs to the technical field of deep neural networks, and relates to a meta-learning algorithm, in particular to a meta-learning algorithm based on gradual gradient correction of a meta-learner. Background technique [0002] Deep neural networks have recently achieved remarkable results in different applications due to their powerful modeling capabilities for complex input patterns. Nevertheless, deep neural networks are prone to overfitting on training data containing noisy annotations, resulting in poor generalization during prediction. In practice, this robust learning problem for noisy annotations is often unavoidable due to the scarcity of high-quality annotations. Typical examples, such as using crowdsourcing systems or search engines for data collection, often generate a large number of wrong labels, resulting in low-quality training data. Therefore, effective learning of data containing noisy labels is a very important and cha...

AI Technical Summary

Problems solved by technology

However, in order to ensure a strong hyperparameter learning ability, current methods generally need to construct very complex meta-learners.

This makes these meta-learning algorithms hard to understand and hard to reproduce

Images