Entity relationship extraction method for automatically constructing data set

Abstract

The invention provides an entity relationship extraction method for automatically constructing a data set. The entity relationship extraction method comprises the following steps: 1, collecting and preprocessing corpora; 2, defining a triple dictionary table, and constructing a synonym table; 3, generating a training data set and a test data set by using an LTP tool; 4, training a network model according to the training data set; 5, performing entity and relationship prediction on the test data set through the trained network model; 6, optimizing the prediction result to obtain a triple data set. By the adoption of the scheme, text content can be automatically analyzed, and the problem that a training and testing data set is difficult to generate is effectively solved; according to the scheme, through optimization and adjustment of the bert model, the problem that extraction of the entity relationship needs to depend on a large number of resource calculations in the past is solved, theentity relationship in the text can be efficiently extracted only through fine adjustment of the bert model, and therefore the intrinsic relationship between the multi-source heterogeneous data is visually presented.

Description

technical field [0001] The invention relates to the field of natural language processing, in particular to an entity relationship extraction method for automatically constructing a data set. Background technique [0002] Entities and relationships summarize the main content of the text, can intuitively display the connection between data, and provide basic data for downstream tasks such as intelligent question answering and retrieval systems; at present, except for a few keywords provided by standardized documents such as papers, most Documents do not provide an intuitive data structure that can reflect their content; the traditional methods of relying on manual reading of texts to extract document entities and relationships are increasingly unable to meet the needs of practical applications in today's multi-source and massive document data; therefore, how to Efficient and accurate extraction of entities and relationships is a problem that needs to be solved urgently. At pre...

AI Technical Summary

Problems solved by technology

This solution mainly solves the following two problems. On the one hand, it can realize automatic parsing of text content, effectively solving the problem of difficult generation of training and test data sets; Extracting problems that rely on a large number of resource calculations, only by fine-tuning the Bert model, can efficiently extract the entity relationship in the text, so as to intuitively present the essential connection between multi-source heterogeneous data

Images