GRU based recurrent neural network multi-label learning method

Abstract

The invention provides a GRU based recurrent neural network multi-label learning method, which comprises the steps of S1, initializing a system parameter [theta]=(W, U, B); S2, inputting a sample {xi,yi}<i=1><N>, calculating a hidden state hT of the output of an RNN (Recurrent Neural Network) at each moment, wherein the sample xi belongs to R<M*1>, yi is a multi-label vector of the sample xi, andyi belongs to R<C*1>; S3, calculating a context vector hT and output zi of an output layer; S4, calculating the predicted output yi^, calculating the loss Li, and determining an objective function J;S5, solving the gradient of the system parameter [theta]=(W, U, B) according to a gradient descent method and a BPTT (Back-propagation Through Time) algorithm; S6, determining a learning rate [eta],and updating each weight gradient W=W-[eta]*[delta]W; S7, judging whether the neural network reaches stable or not, if so, executing a step S8, if not, returning the step S2, and iteratively updatingmodel parameters; and S8, outputting an optimization model. According to the invention, effective feature representation of the sample can be obtained by fully utilizing the RNN so as to improve the accuracy of multi-label classification. In addition, a problem of gradient disappearance is not easy to occur in back propagation.

Description

technical field [0001] The invention relates to the field of recurrent neural network multi-label learning, in particular to a GRU-based recurrent neural network multi-label learning method. Background technique [0002] In the field of machine learning, multi-label classification plays an important role in classification problems. The traditional label classification problem is to learn a single label for an example from a set of labels. Such a problem is called a binary classification problem (or a text and network data filtering problem). Each example in a multi-label classification problem has a set of associated labels. In solving the problem of multi-label classification, many methods have emerged in recent years. The methods are basically divided into two types. One is to convert the problem into a traditional single-label classification. This method learns each label in the label set separately through multiple binary classifiers. The second is to adjust existing ...

AI Technical Summary

Problems solved by technology

[0006] The existing neural network multi-label classification cannot effectively learn the basic characteristics of the sample, the implementation structure is complex, and the gradient disappearance problem is prone to occur during backpropagation

Images