Dependency syntax pre-training model-based chapter-level relation extraction method and system

Abstract

The invention discloses a chapter-level relation extraction method and system based on a dependency syntax pre-training model, and relates to the technical field of natural language processing, dependency syntax information is introduced into the pre-training model by adopting direct conversion, double affine conversion and decomposition linear conversion, characteristics of different introduction modes are compared respectively, and a chapter-level relation is extracted. Analyzing the effect of the dependency syntax information on the pre-training model, and exploring the auxiliary effect of the graph structure on the pre-training model; the entity feature representation and the distance feature between the entities are spliced, so that the information of the entities is enriched, and the subsequent chapter-level relation extraction task is facilitated. According to the method, the problem that the existing pre-training model is difficult to process a long-distance text in text-level relation extraction and is weak in dependence is solved, the long text processing capability and the performance precision of the pre-training model in a text-level relation extraction task are improved, and the auxiliary effect of the graph structure on the pre-training model is explored.

Description

technical field [0001] The invention relates to the technical field of natural language processing, in particular to a method and system for extracting text-level relations based on dependency syntax and training models. Background technique [0002] Relational extraction is to extract unknown relational facts from plain text, which is a very important step in text mining. Relation extraction has many critical real-world applications, such as question answering tasks and text analysis tasks. Most relational extractions only focus on relational facts within a single sentence, and many relational facts do not exist only in a single sentence, but are distributed across multiple sentences. Therefore, the task of relation extraction gradually develops to the text level. [0003] At present, the mainstream chapter-level relationship extraction methods can be roughly divided into three categories: [0004] (1), Based on the design of different levels of feature extractors. This...

AI Technical Summary

Problems solved by technology

[0006] (3) Based on the pre-training model method, although the Transformer-based pre-training model can implicitly solve long-distance dependencies, the attention mechanism does not take into account the two tokens when modeling the dependencies between any two tokens. The position information or other relevant information of each token in the input sequence is one of the reasons why the pre-trained model still has great limitations in processing long texts.

Images