Entity-aware relationship extraction method, device and equipment and storage medium

Abstract

The application discloses an entity-aware relationship extraction method, device and equipment and a storage medium, and steps are as follows: a label sequence is constructed for an entity, and the label sequence is spliced with a text to obtain an input sequence; a mask matrix of the input sequence is constructed; a pre-trained language model is used to encode the input sequence to obtain a text vector sequence; a head vector and a tail vector of a known entity are taken out, spliced and mapped to obtain an entity vector representation; and each entity vector is spliced two by two to predict an entity pair relationship. The entity-aware relationship extraction method of the application, without changing the structure of a pre-trained model, redefines a reserved character of the pre-trained model, combines a mask mechanism and position coding, and fuses multiple entity information at a text coding layer, so that a one-time coding model fusing entity information is realized. Compared with the prior art, the method has a simpler step sequence, higher extraction efficiency, lower requirement for device computing capacity, and good applicability to various pre-trained language models, and has a good application prospect.

Description

Technical Field

[0001] This invention relates to the field of information extraction technology in machine learning and natural language processing, and to a method, apparatus, device, and storage medium for relation extraction based on entity perception. Background Technology

[0002] Since the invention of the internet, countless amounts of data have accumulated online, with unstructured text comprising a significant portion. Automated methods for extracting information from this text would greatly simplify the information acquisition process and improve the efficiency of information collection. While entity recognition technology can identify valuable entities in text, these entities often lack connections and fail to form a cohesive whole. However, by combining this with relation extraction technology, we can establish relationships between entities, using various relational structures to create an entity-relationship network. This provides a more efficient way to express, organize, and utilize heterogeneous, massive amounts of data.

[0003] Relation extraction, by inputting known entities and their corresponding text, determines the relationships between these entities. It connects independently existing entities using various relationships and is a key technology in knowledge graph construction and a fundamental technology in the field of information extraction. In relation extraction, the effective utilization of known entity information and information between different entity pairs significantly impacts the determination of relationships between entity pairs. Currently, most methods struggle to efficiently integrate entity information. CN 113468888 A discloses a method and apparatus for joint entity relation extraction based on neural networks. It utilizes a pre-trained language model ERNIE for random masking, then introduces pre-trained Chinese word vectors (CWV) to achieve word information fusion, performs handshake annotation, decodes the handshake annotation, and uses the UCL national standard for indexing. However, its overall steps are complex, its relation extraction efficiency is low, it requires high computing power, and it is only applicable to the ERNIE pre-trained language model.

[0004] Therefore, developing a relation extraction technique that can efficiently incorporate known entity information, has good applicability, and high extraction efficiency is of great practical significance. Summary of the Invention

[0005] Due to the aforementioned deficiencies in existing technologies, this invention provides a relation extraction technique that can efficiently incorporate known entity information, has good applicability, and high extraction efficiency, overcoming the shortcomings of existing relation extraction techniques, such as poor applicability, low extraction efficiency, and excessive requirements for device computing power.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A relation extraction method based on entity perception includes the following steps:

[0008] 1) Construct a token sequence for the entity and concatenate the token sequence with the text to obtain the input sequence;

[0009] 2) Construct the mask matrix for the input sequence;

[0010] 3) Encode the input sequence using a pre-trained language model to obtain the text vector sequence H. L ;

[0011] 4) Extract the first and last vectors of the known entity, concatenate them, and map them to obtain the entity vector representation;

[0012] 5) Concatenate the entity vectors pairwise to predict entity pair relationships.

[0013] The relation extraction method based on entity perception of the present invention, without changing the structure of the pre-trained model, redefines the reserved characters of the pre-trained model, combines masking mechanism and positional encoding, and integrates multi-entity information at the text encoding layer, realizing a one-time encoding model that integrates entity information. Compared with the prior art, its steps are simpler, the extraction efficiency is higher, the requirements for device computing power are lower, it can be applied to various pre-trained language models, and its applicability is good and has great application prospects.

[0014] As a preferred technical solution:

[0015] As described above, in the relation extraction method based on entity perception, step 1) specifically includes:

[0016] Add marker characters after the original text. The style of the marker characters is type-position. The type is consistent with the entity type. The position is S or E, which indicates that the character marks the start or end position of a certain type of entity. Append all marker characters to the original text sequence to form the original text-marker sequence. Then, set the position encoding of the marker characters to be consistent with the position of the entity in the original text.

[0017] In the entity-aware relation extraction method described above, step 2) involves constructing the mask matrix according to the following principle:

[0018] The characters in the original text other than entities are only visible in the original text. The characters contained in the entities in the original text are visible in the original text as well as their corresponding entity tags. The entity tag characters are visible in the original text as well as the two tag characters that are themselves.

[0019] As described above, in the entity-aware relation extraction method, step 3) specifically involves:

[0020] The original text and the sequence of tags, along with the positional encoding and mask matrix, are input into a pre-trained language model based on Transformer to obtain a sequence of text vectors.

[0021] As described above, in the relation extraction method based on entity perception, step 4) specifically involves:

[0022] Extract the vectors corresponding to the start and end positions of each entity, concatenate the two vectors of each entity, and map them through a fully connected network to obtain the vector representation of each entity.

[0023] As described above, in the entity-aware relation extraction method, step 5) specifically involves:

[0024] The entity vectors are concatenated and mapped pairwise to form the vector representation of each entity pair. The entity pair type is then determined by a classification layer. Entity pairs that do not have a relationship are predicted as NONE.

[0025] This invention provides a relation extraction device, the device comprising:

[0026] The input sequence acquisition module is used to construct a marker sequence for entities and concatenate the marker sequence with text to obtain the input sequence.

[0027] The mask matrix construction module is used to construct the mask matrix of the input sequence;

[0028] The pre-trained language model processing module is used to encode the input sequence using the pre-trained language model to obtain a text vector sequence.

[0029] The entity vector acquisition module is used to extract the first and last vectors of a known entity, concatenate them, and map them to obtain the entity vector representation;

[0030] The entity pair relationship prediction module is used to concatenate the entity vectors pairwise to predict entity pair relationships.

[0031] The present invention also provides a computer device, the computer device comprising:

[0032] At least one processor; and,

[0033] A memory communicatively connected to the at least one processor; wherein,

[0034] The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the entity-aware relation extraction method described above.

[0035] Furthermore, the present invention provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the entity-aware relation extraction method described above.

[0036] The above technical solution is only one feasible technical solution of the present invention. The scope of protection of the present invention is not limited thereto. Those skilled in the art can reasonably adjust the specific design according to actual needs.

[0037] The above invention has the following advantages or beneficial effects:

[0038] (1) The relation extraction method based on entity perception of the present invention, without changing the structure of the pre-trained model, redefines the reserved characters of the pre-trained model, combines the masking mechanism and position encoding, and integrates multiple entity information at the text encoding layer, thereby realizing a one-time encoding model that integrates entity information.

[0039] (2) The entity perception-based relation extraction method of the present invention has a simpler step sequence and higher extraction efficiency (high accuracy and model efficiency) compared with the prior art. It has lower requirements for device computing power and can be applied to various pre-trained language models. It has good applicability and great application prospects. Attached Figure Description

[0040] The invention, its features, shape, and advantages will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. Like reference numerals denote like parts throughout the drawings. The drawings are not drawn to scale; the emphasis is on illustrating the gist of the invention.

[0041] Figure 1 This is an overall flowchart of the entity-aware relation extraction method of the present invention;

[0042] Figure 2 This is a diagram illustrating a relation extraction example;

[0043] Figure 3 This is a schematic diagram of entity marking and location encoding;

[0044] Figure 4 This is a schematic diagram of the mask matrix design;

[0045] Figure 5 This is a schematic diagram of the relation extraction device of the present invention;

[0046] Figure 6 This is a schematic diagram of the structure of the computer device of the present invention;

[0047] Among them, 1-relation extraction device, 11-input sequence acquisition module, 12-mask matrix construction module, 13-pre-trained language model processing module, 14-entity vector acquisition module, 15-entity pair relation prediction module, 2-computer equipment, 21-processor, 22-memory. Detailed Implementation

[0048] The structure of the present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the present invention.

[0049] Example 1

[0050] A relation extraction method based on entity perception, the steps of which are as follows: Figure 1 As shown, specifically:

[0051] S1: Construct a marker sequence for the entity and concatenate the marker sequence with the text to obtain the input sequence;

[0052] S2: Construct the mask matrix of the input sequence;

[0053] S3: Encode the input sequence using a pre-trained language model to obtain the text vector sequence H. L ;

[0054] S4: Extract the first and last vectors of the known entity, concatenate them, and map them to obtain the entity vector representation;

[0055] S5: Concatenate the entity vectors pairwise to predict entity pair relationships.

[0056] Step S1 is as follows:

[0057] For a text containing N entities (where the text and its entity relationships are as follows) Figure 2 As shown), each entity uses a pair of tags <type-S, type-E>, resulting in 2N tags. These 2N tags are directly appended to the original text to form a new text sequence. Then, positional encoding is performed, while the original text retains its original positional encoding. The positional encoding of the entity tags is consistent with the entity corresponding to each tag, as shown below. Figure 3 As shown.

[0058] Step S2 is as follows:

[0059] A mask matrix is ​​constructed between the original text and the labeled sequence to control their computational interaction range in the pre-trained language model encoding. The design principle of the mask matrix is ​​that words in the original text, excluding entities, can only see words in the original text; words contained in entities in the original text can see words in the original text and their corresponding entity labels; entity label characters can see words in the original text and their two corresponding label characters, such as... Figure 4 As shown.

[0060] Step S3 is as follows:

[0061] The sequence of original text and labeled data, along with positional encoding and a mask matrix, is input into a pre-trained language model to obtain its top-level vector sequence H. L .

[0062] Step S4 is as follows:

[0063] The start and end positions of N entities are mapped to the vector sequence H. L The corresponding vectors are concatenated and mapped to the entity representation vector space through a fully connected network to obtain vector representations of N entities.

[0064] Step S5 is as follows:

[0065] The N entity vectors are distinguished by their pairings before and after, resulting in N(N-1) / 2 entity pair vectors. The entity pair vectors are then used to predict the relationship between the entity pairs through a classification layer. For entity pairs that do not have a relationship, they are predicted as NONE.

[0066] At this point, relation extraction based on entity perception has been completed.

[0067] The entity-aware relation extraction method provided in this invention integrates the position and type information of multiple entities into the encoding layer (pre-trained language model) without changing the pre-trained language model. This eliminates the need for large-scale retraining of the pre-trained language model and enables the classification of multiple entity pairs with a single encoding.

[0068] Example 2

[0069] A relation extraction device 1, the device as follows Figure 5 As shown, it includes:

[0070] The input sequence acquisition module 12 is used to construct a marker sequence for entities and concatenate the marker sequence with text to obtain the input sequence.

[0071] Mask matrix construction module 12 is used to construct the mask matrix of the input sequence;

[0072] The pre-trained language model processing module 13 is used to encode the input sequence using the pre-trained language model to obtain a text vector sequence.

[0073] The entity vector acquisition module 14 is used to extract the first and last vectors of a known entity, concatenate them, and map them to obtain an entity vector representation;

[0074] The entity pair relationship prediction module 15 is used to concatenate the entity vectors pairwise to predict the entity pair relationship.

[0075] Example 3

[0076] A computer device, computer device 2 such as Figure 6 As shown, it includes: at least one processor 21; and a memory 22 communicatively connected to at least one processor 21; wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the entity-aware relation extraction method as described in Embodiment 1.

[0077] Example 4

[0078] A computer-readable storage medium stores computer-readable instructions that, when executed by a processor, implement the entity-aware relation extraction method as described in Embodiment 1.

[0079] Those skilled in the art should understand that variations can be implemented by combining existing technology with the above embodiments, which will not be elaborated here. Such variations do not affect the essence of the present invention and will not be elaborated here.

[0080] The preferred embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and the devices and structures not described in detail should be understood as being implemented in a conventional manner in the art. Any person skilled in the art can make many possible variations and modifications to the technical solutions of the present invention using the methods and techniques disclosed above, or modify them into equivalent embodiments with equivalent changes, without departing from the scope of the present invention. This does not affect the essential content of the present invention. Therefore, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the present invention's technical solutions still fall within the protection scope of the present invention.

Claims

1. A relation extraction method based on entity perception, characterized in that, Includes the following steps: 1) Construct a token sequence for the entity and concatenate the token sequence with the text to obtain the input sequence; 2) Construct the mask matrix for the input sequence; 3) Encode the input sequence using a pre-trained language model to obtain the text vector sequence H. L ; 4) Extract the first and last vectors of the known entity, concatenate them, and map them to obtain the entity vector representation; 5) Concatenate the entity vectors pairwise to predict entity pair relationships; Step 1) specifically refers to: Add marker characters after the original text. The style of the marker characters is type-position. The type is consistent with the entity type. The position is S or E, which indicates that the character marks the start or end position of a certain type of entity. Append all marker characters to the original text sequence to form the original text-marker sequence. Then set the position encoding of the marker characters to be consistent with the position of the entity in the original text. In step 2), the principle for constructing the mask matrix is ​​as follows: The characters in the original text other than entities are only visible in the original text. The characters contained in the entities in the original text are visible in the original text as well as their corresponding entity tags. The entity tag characters are visible in the original text as well as the two tag characters that are themselves.

2. The relation extraction method based on entity perception according to claim 1, characterized in that, Step 3) specifically refers to: The original text and the sequence of tags, along with the positional encoding and mask matrix, are input into a Transformer-based pre-trained language model to obtain the text vector sequence H. L .

3. The relation extraction method based on entity perception according to claim 1, characterized in that, Step 4) specifically involves: Extract the vectors corresponding to the start and end positions of each entity, concatenate the two vectors of each entity, and map them through a fully connected network to obtain the vector representation of each entity.

4. The relation extraction method based on entity perception according to claim 1, characterized in that, Step 5) specifically involves: The entity vectors are concatenated and mapped pairwise to form the vector representation of each entity pair. The entity pair type is then determined by a classification layer. Entity pairs that do not have a relationship are predicted as NONE.

5. A relation extraction device, characterized in that, The device includes: The input sequence acquisition module is used to construct a marker sequence for entities and concatenate the marker sequence with the text to obtain the input sequence. Specifically, it adds marker characters after the original text. The style of the marker characters is type-position. The type is consistent with the entity type. The position is S or E, which indicates that the character marks the start or end position of a certain type of entity. All marker characters are appended to the original text sequence to form the original text-marker sequence. Then, the position encoding of the marker characters is set to be consistent with the position of the entity in the original text. The mask matrix construction module is used to construct a mask matrix for the input sequence. The construction principle of the mask matrix is ​​as follows: words in the original text other than entities are only visible to words in the original text; words contained in entities in the original text are visible to words in the original text and their corresponding entity tags; entity tag characters are visible to words in the original text and their two tag characters. The pre-trained language model processing module is used to encode the input sequence using a pre-trained language model to obtain a text vector sequence H. L ; The entity vector acquisition module is used to extract the first and last vectors of a known entity, concatenate them, and map them to obtain the entity vector representation; The entity pair relationship prediction module is used to concatenate the entity vectors pairwise to predict entity pair relationships.

6. A computer device, characterized in that, The computer device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the entity-aware relation extraction method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the entity-aware relation extraction method as described in any one of claims 1 to 4.

Images