Efficient lifetime relation extraction method and system based on dynamic regularization

Abstract

The present invention provides a method and system for extracting efficient lifelong relationships based on dynamic regularization. The method includes the following steps: receiving multiple data sets, using a neural model to sequentially train training samples in the data sets, and each data set corresponds to a task; Create a memory block to store memory data for the trained data set, and access the memory data of all memory blocks when training a new data set. To overcome catastrophic forgetting, define memory data loss function, define feature loss function, and define EWC loss Function; establish the training loss difference model during continuous training tasks to calculate the regularization factor of the feature loss function, memory data loss function, and EWC loss function respectively; according to the feature loss function and its regularization factor, memory data loss function and its regularization factor, EWC loss function and its regularization factor to obtain the optimal relation extraction parameters. This method can achieve higher accuracy when extracting lifetime relationships without introducing additional parameters.

Description

technical field [0001] The invention belongs to the field of computer natural language processing, and in particular relates to an efficient lifelong relationship extraction method and system based on dynamic regularization. Background technique [0002] Relation extraction aims to identify the relationship facts of paired entities in text, which can be applied to many natural language processing fields, such as knowledge base construction and question answering systems. Compared with traditional methods that focus on manually designed features, today's CNN-based (convolution neural network) or RNN (recurrent neural network) neural network methods have achieved impressive progress in relation extraction, but most neural models use methods that assume a set of predetermined relations, but this assumption does not Always applicable in practical relation extraction scenarios. [0003] Existing research on lifelong learning is dedicated to overcoming the phenomenon of catastrop...

AI Technical Summary

Problems solved by technology

[0003] Existing research on lifelong learning is dedicated to overcoming the phenomenon of catastrophic forgetting, dealing with lifelong learning of neural network models (also known as continuous learning). However, this heuristic method requires storing all previous training data as well as new data to train a completely new model, which is expensive and time-consuming

[0004] Therefore, the goal of lifelong learning in relation extraction is to make the model perform well on a series of tasks, thus avoiding revisiting all previous data at each stage, but most existing methods are designed for a fixed set of relations. designed, they cannot adapt the trained model to the newly added relationship without catastrophically forgetting the previously learned knowledge, the phenomenon of catastrophic forgetting, which refers to the significant performance drop of the model when switching from the old task to the new task

To alleviate the forgetting problem, it has been proposed to use regularization terms to prevent drastic changes in parameter values ​​while still being able to find good solutions for new tasks, or to augment models with episodic memory modules, which have been used in simple image classification datasets. obtained considerable performance gains on , but it turns out that they perform poorly in natural language processing scenarios

In fact, there is only limited literature discussing lifelong learning on NLP tasks such as relation extraction; to fill the gap in this field, Wang, H., 2019 proposed a method to overcome the forgetting problem of relation extraction models , they introduced an explicit alignment model to alleviate the distortion of the sentence embedding space when the model learns new data, and achieved the best performance, but although this method can work effectively, it is very dependent on the use of the alignment model , which introduces more parameters into an already over-parameterized relation extraction model, which leads to an increase in the number of supervisory signals, memory and computational resources required for training

Images