A Scalable Neural Network Based Sequence Labeling Method

Abstract

The invention discloses an extensible sequence labeling method based on a neural network. The method includes the steps of creating a laminated nth-order model, and utilizing the laminated nth-order model to carry out label prediction on a set sequence to obtain a label sequence. The training process of the laminated nth-order model includes the steps of firstly, generating n label sets accordingto labels in each labeling unit in a training corpus, wherein the n label sets comprise the first-order label set, the second-order label set... and the nth-order label set; combining the label of a labeling unit i with the label of an adjacent n-1 labeling unit to form an nth-order label of the labeling unit i, wherein the nth-order label set is a label set composed of nth-order labels of each labeling unit; then, utilizing all the obtained label sets different in order to train the neural network separately to obtain n models, wherein the n models include the first-order neural network model, the second-order neural network mode ... and the nth-order neural network model. The extensible sequence labeling method can obviously reduce the overfitting risks of models and improve the effect of a sequence labeling task.

Description

technical field

[0001] The invention belongs to the field of natural language processing and relates to sequence labeling, in particular to a sequence labeling method for combined decoding of different order model information. Background technique

[0002] When the neural network deals with the problem of sequence labeling, in the training phase, its corresponding label is predicted for each. The cost function is the cross entropy between the predicted output of the neural network and the standard label, and the training process minimizes the objective function. In the decoding stage, the label of the current word is directly predicted by the neural network.

[0003] When the existing neural network deals with the sequence labeling problem, the label predicted for the current word (word) does not involve the surrounding word (word) labels, that is, the predicted label of each word (word) is independent of other words (words) , and then perform gradient descent on the basis ...

Images