Scientific and technological resource relation extraction method and device based on pre-training language model

Abstract

The invention discloses a scientific and technological resource relation extraction method and device based on a pre-training language model. The method comprises the following steps: inputting science and technology resource statements to be subjected to relation classification into a science and technology resource relation extraction model trained based on a pre-training language model; outputting a result by the scientific and technological resource relation extraction model, the result being a relation category between the two entities in the scientific and technological resource statement. According to the pre-training language model based on massive unsupervised data training, more efficient language feature learning ability can be provided, and meanwhile, the polysemy problem caused by using external pre-training word vectors is avoided. Moreover, in combination with a metric learning method, a scientific and technological resource relationship extraction problem is combined with a small sample learning problem, so that a relatively strong relationship extraction capability is realized by learning a small amount of training data, and a good relationship classification effect can be achieved under the condition that annotation data is relatively deficient.

Description

technical field [0001] The invention relates to a method for extracting scientific and technological resource relations based on a pre-trained language model, and also relates to a corresponding scientific and technological resource relation extraction device, which belongs to the technical field of natural language processing. Background technique [0002] In the process of constructing the knowledge map of scientific and technological resources, it is necessary to extract useful knowledge from a large number of texts, mainly the entities in the text and the relationship between entities. Named entity recognition is a subtask of information extraction, which aims to identify entities with specific meanings from text data, usually including names of people, places, institutions, domain names, etc. Relation extraction is after identifying related entities in the text, the information extraction task also needs to extract the semantic relationship between any two entities, for...

AI Technical Summary

Problems solved by technology

Traditional machine learning models rely heavily on the quality of features, and feature engineering is time-consuming and laborious

The effect of the deep learning model is affected by the size of the training data. However, the size of the data sets in the field of relationship extraction is small, and the workload of accurately labeling the training data is also huge.

Images