Unlock instant, AI-driven research and patent intelligence for your innovation.

A weakly supervised learning method for anaphora resolution using language models

A technology of referring to resolution and language models, applied in neural learning methods, biological neural network models, natural language data processing, etc., can solve problems such as decreased accuracy, improve accuracy, improve interpretability, and be widely used sexual effect

Active Publication Date: 2021-05-18
BEIJING INSTITUTE OF TECHNOLOGYGY
View PDF11 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to propose a weakly supervised learning method for anaphora resolution using a language model in view of the technical defect that the accuracy of the existing anaphora resolution method is affected by the lack of data.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A weakly supervised learning method for anaphora resolution using language models
  • A weakly supervised learning method for anaphora resolution using language models
  • A weakly supervised learning method for anaphora resolution using language models

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0079] This embodiment illustrates the specific implementation of a weakly supervised method for anaphora resolution using a language model according to the present invention.

[0080] figure 1 Shown is the flow chart of the method. During the training process, sentences are randomly extracted from labeled and unlabeled data in turn to input the model.

[0081] In practice, unlabeled data is often large-scale; small-scale data refers to training text chapters containing thousands of orders, that is, there are thousands of texts in the data, and each text is about a few hundred words in length; large-scale Data means that the text in the data is more than one million, and the length of each text is also about a few hundred words;

[0082] The marked data already contains the results of word segmentation and part of speech manually marked, so only word vector generation is performed on it.

[0083] figure 2 Shown is the calculation process of the three losses included in the...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a weakly supervised learning method for anaphora resolution using a language model, and belongs to the technical field of information extraction in natural language processing. The method includes: step 1: preprocessing of the data set; step 2: first training the anaphora resolution model on a small amount of labeled data set; step 3: training language based on a multi-head self-attention mechanism on a large-scale unlabeled data set model; Step 4: Perform weakly supervised learning based on the output of the anaphora resolution model on unlabeled and labeled data, introduce a loss specially designed for the multi-head self-attention mechanism, and divide the taps in the multi-head self-attention mechanism into special Different losses are computed for taps and common taps, respectively. The method enables the special tap to have the ability to output a distribution probability similar to that of the anaphora resolution model, which improves the accuracy of the anaphora resolution system, and the obtained language model and anaphora resolution model can expand the field of use of the existing anaphora resolution model, Model parameters have better interpretability.

Description

technical field [0001] The invention relates to a weakly supervised learning method for anaphora resolution using a language model, and belongs to the technical field of natural language processing. Background technique [0002] Anaphora resolution refers to analyzing all the words in the text that represent the same entity in a given text, which is usually a person or an object. Take the sentence "Xiao Li went to France for a trip and he had a great time there." For example, the two pronouns "he" and "over there" in the sentence represent "Xiao Li" and "France" respectively. The relationship between the pronouns "he" and "Xiao Li" is called "sufficiency", and words with exact meaning in this "sufficiency" relationship are called antecedents, and antecedents are usually noun phrases, that is, "Xiao Li" in the example sentence " and "France"; words whose expression changes with the antecedent are called anaphors, and anaphors are usually pronouns, that is, "he" and "there" i...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F40/284G06F40/295G06F40/247G06N3/04G06N3/08
CPCG06N3/084G06N3/047
Inventor 辛欣明坤
Owner BEIJING INSTITUTE OF TECHNOLOGYGY