RNN-based multi-task learning method

Abstract

The invention provides an RNN-based multi-task learning method. The method includes steps of S1, initializing a system parameter theta=(W, U, B, V); S2, inputting samples x<1, i>... x<R, i>, learningpublic information Xco and compensating the public information into training of single tasks; S3, calculating predication label vector output, as in the description, of each neural network and calculating loss Lr of a task <r, i>; S4, solving the gradient of theta=(W, U, B, V) according to a gradient decrease method and a BPTT algorithm and determining the gradient of a task r relative to the public information Xco; S5, determining a learning rate eta and updating each weight gradient W=W-eta*deltaW; S6, judging whether a neural network reaches stability or not, if yes, performing a step S7 and if not, returning to step S2 and performing iteration update on model parameters; S7, outputting an optimized model. According to the invention, public features among RNN learning multiple tasks canbe utilized effectively and the public features are input to learning of the single tasks, so that information share is realized. Besides, through introduction of a GRU structure in RNN, the problemof gradient vanishment can be solved effectively.

Description

technical field [0001] The invention relates to the field of neural network multi-task learning, in particular to an RNN-based multi-task learning method. Background technique [0002] In real-world applications, different tasks can be related to each other in different ways. And multi-task learning has more advantages than single-task learning. For example, when we only have a small part of the available data for each task, then multi-task learning can combine the data of multiple related tasks for learning. It is also possible that tasks are linked by some underlying common representation. For example, in object recognition, the first few steps in the formation of the human visual system are all by learning a common set of features to represent all objects. Most of the previous methods on multi-task learning link the relationship between tasks through a functional concept. [0003] For the modeling of sequence data, the model based on neural network (Neural Network) ha...

AI Technical Summary

Problems solved by technology

Most of the existing multi-task learning methods have complex structures, such as LSTM models, and are prone to gradient disappearance in the process of backpropagation. Based on this, the present invention provides a multi-task learning method based on RNN. The RNN has a GRU structure , which can effectively prevent the gradient disappearance problem, and is simpler than the LSTM structure, making the obtained features more accurate

Images