Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A Scalable Neural Network Based Sequence Labeling Method

A neural network and sequence labeling technology, applied in the field of natural language processing, to improve the effect, reduce the risk of model overfitting, and reduce the training time.

Active Publication Date: 2019-11-26
PEKING UNIV
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In order to overcome the label-independent technical deficiency of the existing neural network on the sequence labeling problem, the present invention provides a new training and decoding method for the sequence labeling problem that is easy to expand (in the sequence labeling problem of the neural network, the decoding process is to obtain The process of entering a label sequence)

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Scalable Neural Network Based Sequence Labeling Method
  • A Scalable Neural Network Based Sequence Labeling Method
  • A Scalable Neural Network Based Sequence Labeling Method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] Below in conjunction with accompanying drawing, further describe the present invention by example, but do not limit the scope of the present invention in any way.

[0030] The invention provides a new neural network model training and decoding method for sequence labeling. figure 1 It is a schematic diagram of the training method of the traditional neural network model. Such as figure 1 As shown, each standard label in the model only involves its own label, which is a single-stage model.

[0031] figure 2 It is the neural network training method of the present invention, which adopts a new labeling mode. Such as figure 2 As shown, the n-order label of a word is the original label of n words combined as a new label. Due to the change in labeling schema, the entire dataset label set has also changed. Generally speaking, the n-order label set contains all possible combinations of n first-order labels, which is equivalent to performing n-1 Cartesian products on the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses an extensible sequence labeling method based on a neural network. The method includes the steps of creating a laminated nth-order model, and utilizing the laminated nth-order model to carry out label prediction on a set sequence to obtain a label sequence. The training process of the laminated nth-order model includes the steps of firstly, generating n label sets accordingto labels in each labeling unit in a training corpus, wherein the n label sets comprise the first-order label set, the second-order label set... and the nth-order label set; combining the label of a labeling unit i with the label of an adjacent n-1 labeling unit to form an nth-order label of the labeling unit i, wherein the nth-order label set is a label set composed of nth-order labels of each labeling unit; then, utilizing all the obtained label sets different in order to train the neural network separately to obtain n models, wherein the n models include the first-order neural network model, the second-order neural network mode ... and the nth-order neural network model. The extensible sequence labeling method can obviously reduce the overfitting risks of models and improve the effect of a sequence labeling task.

Description

technical field [0001] The invention belongs to the field of natural language processing and relates to sequence labeling, in particular to a sequence labeling method for combined decoding of different order model information. Background technique [0002] When the neural network deals with the problem of sequence labeling, in the training phase, its corresponding label is predicted for each. The cost function is the cross entropy between the predicted output of the neural network and the standard label, and the training process minimizes the objective function. In the decoding stage, the label of the current word is directly predicted by the neural network. [0003] When the existing neural network deals with the sequence labeling problem, the label predicted for the current word (word) does not involve the surrounding word (word) labels, that is, the predicted label of each word (word) is independent of other words (words) , and then perform gradient descent on the basis ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/21G06N3/02
CPCG06F40/117G06N3/02
Inventor 孙栩张艺杨洋
Owner PEKING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products