Entity relationship joint extraction method and system based on interactive bi-affine mechanism

Abstract

The application belongs to the technical field of data mining, and provides an entity relationship joint extraction method and system based on an interactive double-affine mechanism.The application divides the entity relationship joint extraction task of text data into three subtasks of subject entity identification, object entity identification and subject-object pair relationship classification which can be jointly learned.In the entity identification and relationship classification stage, through a preset double-affine mechanism model, the subject, the object and the subject-object pair relationship can be respectively mapped into a subject entity table, an object entity table and a subject-object pair relationship classification table.The subject entity identification, the object entity identification and the subject-object pair relationship classification can further interact bidirectionally at the subtask level, improve the entity relationship extraction effect, and solve the problems of error propagation, entity redundancy and insufficient information interaction between subtasks in the identification and classification process.

Description

Technical Field

[0001] This invention belongs to the field of data mining technology, and in particular relates to a method and system for joint extraction of entity relations based on an interactive dual affine mechanism. Background Technology

[0002] With the emergence and rapid development of the internet, a vast amount of unstructured text data is generated daily online in the form of news articles, research publications, blogs, platform comments, and social media information. Therefore, developing technologies to automatically extract information from this unstructured text data is crucial, as much important and usable information is hidden within it. Entity relation joint extraction, a core task of text mining and information extraction, primarily involves modeling text information to automatically extract entities and the semantic relationships between entity pairs. This extracted data is stored as <subject entity, relation type, object entity> triples, and the corresponding knowledge base is expanded to better support applications such as intelligent question answering and intelligent retrieval.

[0003] The inventors discovered that, depending on the order in which the two subtasks of entity recognition and relation classification are completed, supervised entity-relation extraction methods based on deep learning can be divided into pipelined methods and joint learning methods. Pipeline methods treat entity-relation extraction as two separate subtasks of entity recognition and relation classification, but are prone to errors such as error propagation, entity redundancy, and insufficient information interaction between subtasks. Joint learning methods, on the other hand, model these two subtasks simultaneously, effectively integrating entity and relation information, simultaneously extracting entities and classifying relations between entity pairs. This effectively solves the problems of error propagation and information interaction between the two subtasks in pipelined methods. However, in joint learning methods, the information interaction between the entity recognition and relation classification subtasks only occurs in the input sharing layer; the interaction between the two specific tasks is still insufficient. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes a joint entity relation extraction method and system based on an interactive dual-affine mechanism. Building upon parallel encoding, this invention further decomposes the joint entity relation extraction task into three jointly learnable sub-tasks: subject entity recognition, object entity recognition, and subject-object pair relation classification. In the entity recognition and relation classification stages, a dual-affine mechanism is introduced, allowing the subject, object, and relation to be mapped to three different tables for joint learning. In the relation classification stage, a shared convolutional neural network (CNN) is introduced, enabling bidirectional interaction between subject entity recognition, object entity recognition, and subject-object pair relation classification at the sub-task level. This scheme achieves good results on the publicly available NYT and WEBNLG relation extraction datasets.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0006] In a first aspect, the present invention provides a method for joint extraction of entity relations based on an interactive dual affine mechanism, comprising:

[0007] Get text data;

[0008] Tokenize the acquired text data;

[0009] The task of jointly extracting entity relations from text data is decomposed into three joint learning sub-tasks: subject entity recognition, object entity recognition, and subject-object pair relation classification.

[0010] Based on the pre-defined dual affine mechanism model, the tokenized text data is mapped to the subject entity table, the object entity table, and the subject-object pair relationship classification table;

[0011] The entity relationships are extracted from the combined subject entity table, object entity table, and subject-object pair relationship classification table.

[0012] Furthermore, the acquired text data is tokenized using a pre-trained language model and processed into token form. During model training, for each line of data in the training set, based on the given subject entity, subject-object pair relation, and object entity triple form, each subject entity is first processed into token form. Then, based on the subject entity token, the start index and end index of the entity in the sentence token are found as the span of the subject entity in the sentence.

[0013] Furthermore, the tokenized text data is encoded to obtain a hidden vector representation of each word in the sentence with contextual information.

[0014] The encoded hidden vector representation is transformed to obtain the hidden vector of each token as the main representation and the hidden vector of the main representation;

[0015] The hidden vectors of the obtained subject representations are encoded to obtain the hidden vectors of the start and end positions of each subject representation.

[0016] Using a dual affine mechanism model, the hidden vectors at the start and end positions of all the main entities are mapped to obtain the main entity table.

[0017] Furthermore, the encoded hidden vector representation is transformed to obtain the hidden vector of each token as the object representation and the hidden vector of the object representation;

[0018] The hidden vectors of the obtained object representations are encoded to obtain the hidden vectors of the start and end positions of each object representation as the object entity.

[0019] Using a dual affine mechanism model, the hidden vectors of the start and end positions of all obtained object entities are mapped to obtain an object entity table.

[0020] Furthermore, in the convolutional neural network with shared parameters, the contextual information representation of the sentence span is extracted, and the span information is fused into the representation of a single token to obtain the subject vector representation and the object vector representation;

[0021] Using a dual affine mechanism model, the obtained subject vector representation and object vector representation with span information are mapped to obtain a subject-object pair relationship classification table.

[0022] Furthermore, each element in the subject entity table, object entity table, and subject-object pair relationship classification table is compared with a preset threshold to obtain the subject entities, object entities, and subject-object pairs relationships that meet the conditions.

[0023] Furthermore, the loss functions include the subject entity recognition loss function, the object entity recognition loss function, and the subject-object pair relationship classification loss function; during model training, the goal is to minimize the sum of the subject entity recognition loss function, the object entity recognition loss function, and the subject-object pair relationship classification loss function.

[0024] Secondly, the present invention also provides an entity relation joint extraction system based on an interactive dual affine mechanism, comprising:

[0025] The data acquisition module is configured to acquire text data.

[0026] The tokenization module is configured to tokenize the acquired text data.

[0027] The task decomposition module is configured to decompose the task of jointly extracting entity relations from text data into three joint learning sub-tasks: subject entity recognition, object entity recognition, and subject-object pair relation classification.

[0028] The mapping module is configured to map the tokenized text data to the subject entity table, the object entity table, and the subject-object pair relationship classification table based on a preset dual affine mechanism model.

[0029] The extraction module is configured to extract entity relationships from the jointly obtained subject entity table, object entity table, and subject-object pair relationship classification table.

[0030] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the entity relation joint extraction method based on the interactive dual affine mechanism described in the first aspect.

[0031] Fourthly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the entity relation joint extraction method based on the interactive dual affine mechanism described in the first aspect.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0033] This invention decomposes the joint entity relation extraction task of text data into three sub-tasks that can be jointly learned: subject entity recognition, object entity recognition, and subject-object pair relation classification. In the entity recognition and relation classification stages, through a pre-set dual affine mechanism model, the subject, object, and subject-object pair relations can be mapped to the subject entity table, object entity table, and subject-object pair relation classification table, respectively. Subject entity recognition, object entity recognition, and subject-object pair relation classification can further interact bidirectionally at the sub-task level, improving the entity relation extraction effect. At the same time, it solves the problems of error propagation, entity redundancy, and insufficient information interaction between sub-tasks that are easy to occur during the recognition and classification process. Attached Figure Description

[0034] The accompanying drawings, which form part of this embodiment, are used to provide a further understanding of this embodiment. The illustrative embodiments and their descriptions are used to explain this embodiment and do not constitute an improper limitation of this embodiment.

[0035] Figure 1 This is a diagram of the interactive dual affine entity relation joint extraction model of Embodiment 1 of the present invention. Detailed Implementation

[0036] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0037] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0038] Example 1:

[0039] As described in the background section, in pipelined methods, entity recognition and relation classification are performed sequentially, leading to problems such as error propagation, redundant entity information, and one-way interaction between the two subtasks. Previous joint learning methods only encoded the entity recognition and relation classification subtasks in the input sharing layer, without sufficient interaction at the task level, resulting in inadequate information exchange between the two subtasks. To address these issues, this embodiment provides an entity relation joint extraction method based on an interactive dual-affine mechanism, comprising: acquiring text data; tokenizing the acquired text data; decomposing the entity relation joint extraction task of the text data into three joint learning subtasks: subject entity recognition, object entity recognition, and subject-object pair relation classification; mapping the tokenized text data to a subject entity table, an object entity table, and a subject-object pair relation classification table according to a preset dual-affine mechanism model; and extracting entity relations from the jointly obtained subject entity table, object entity table, and subject-object pair relation classification table. It is understood that the acquired text data is unstructured text data, such as news articles, research publications, blogs, platform comments, and social media information.

[0040] Specifically, in this embodiment, based on parallel encoding, the joint entity relation extraction task is further decomposed into three jointly learnable sub-tasks: subject entity recognition, object entity recognition, and subject-object pair relation classification. In the entity recognition and relation classification stages, a dual affine mechanism is introduced. This model maps subjects, objects, and relations to three different tables for joint learning. Simultaneously, a shared CNN network is introduced in the relation classification stage, enabling bidirectional interaction between subject entity recognition, object entity recognition, and subject-object pair relation classification at the sub-task level. In other embodiments, the proposed joint entity relation extraction method based on an interactive dual affine mechanism is validated. On the publicly available NYT and WEBNLG relation extraction datasets, the proposed method achieves good results. The specific content of the joint entity relation extraction method based on an interactive dual affine mechanism proposed in this embodiment includes:

[0041] S1. Preprocess the acquired unstructured text data:

[0042] The acquired text data is tokenized. Sentences in the text data are processed into tokens using the Tokenizer tool of the BERT (Bidirectional Encoder Representations from Transformers) pre-trained language model. For each line of data in the training set, based on the given triples in the form of <subject entity, relation type, object entity>, the subject entity is first processed into a token using the Tokenizer. Then, based on the subject entity token, the start and end indices of the entity in the sentence token are found as the span of that subject entity in the sentence. This results in a subject entity table of [sentence token length * sentence token length]. The span [start index, end index] of each entity in the subject entity table is then marked as 1, and other positions in the table are marked as 0. Similarly, each relation triple in the sentence can be represented using a relation table of [sentence token length * sentence token length]. Based on the given original triples, the relation triples can be represented as [subject start index, object start index, relation type]. The [subject start index, object start index] positions in the relation table are then marked as "relation type".

[0043] S2, Encoding:

[0044] In the encoding module, the sentence token obtained after preprocessing in step S1 is input into the interactive dual affine mechanism entity relation joint extraction model. The encoding layer of the model uses the BERT model and loads the weights of the BERT pre-trained model. After the sentence is processed by the encoding layer, each word in the sentence obtains a hidden vector representation with contextual information.

[0045] H enc (X) = BertEncoder(X) (1)

[0046] in, d represents the embedding dimension, n represents the number of tokens, and h represents the hidden vector representation of each token containing context information obtained after the sentence is learned by the encoding layer.

[0047] S3, Decoding:

[0048] The decoding module is divided into a subject entity recognition module, an object entity recognition module, and a subject-object pair relationship classification module. After the encoding layer in step S2 obtains the hidden vector representation corresponding to each token, two independent feedforward neural networks are first applied to transform the vector representation obtained by the encoding layer, respectively obtaining the hidden vectors for each token as the subject representation and the object representation:

[0049] h s (i)=FFNN s (h i (2)

[0050] h o (i)=FFNN o (h i (3)

[0051] in, The obtained subject and object representations will be used in the subsequent subject entity recognition module, object entity recognition module, and subject-object pair relationship classification module.

[0052] S4. After obtaining the hidden vector of the subject representation, two independent feedforward neural networks are applied again in the subject entity recognition module to encode the obtained subject representation hidden vector, respectively obtaining the hidden vector of each subject representation as the starting position of the subject entity. and the hidden vector at the end position

[0053]

[0054]

[0055] Then, a double affine mapping is applied to all the subject start position representations and subject end position representations in the entire sentence to obtain a subject entity table of shape l×l×c, where l represents the sentence length and c represents the number of entity categories. The calculation formula for each element in the obtained subject entity table is as follows:

[0056] t s (i)=σ(h s-start (i) T U s h s-end (i)+W s (h s-start (i)⊕h s-end (i))+b s (6)

[0057] Wherein, s-start and s-end represent the start and end positions of the main entity's span; and b represents trainable parameters; s σ represents the bias parameter; σ represents the sigmoid activation function.

[0058] S5. Similar to the main entity recognition module, two independent feedforward neural networks are also used to transform the obtained object representation hidden vectors, respectively obtaining the hidden vector for each object representation as the starting position of the object entity. and the hidden vector at the end position

[0059]

[0060]

[0061] Similarly, by applying a double affine mapping mechanism to all object start and end position representations in the entire sentence, an l×l×c object entity table is obtained. The calculation formula for each element in the table is as follows:

[0062]

[0063] Where o-start and o-end represent the start and end positions of the object entity's span; and b represents trainable parameters; o σ represents the bias parameter; σ represents the sigmoid activation function.

[0064] S6. In the subject-object pair relationship classification module, a CNN network with shared parameters is first introduced to further extract features of the subject and object representations of the sentence, thereby further enhancing the subject and object representations through bidirectional information interaction between them.

[0065]

[0066]

[0067] In the shared CNN, the kernel sizes are set to [3, 5, 7, 9] to further extract contextual information representations of sentence spans, fusing the span information into the representation of a single token. The final subject vector representation and object vector representation are as follows: and Finally, a biaffine mapping mechanism is applied to the obtained subject token representations and object token representations with span information to obtain an l×l×k subject-object pair relation table, where l represents the sentence length and k represents the predefined number of relation categories. The calculation formulas for each element in the obtained relation table are as follows:

[0068]

[0069] Where s' and o' represent the starting positions of the entire span of the subject and object entities; and b represents trainable parameters; r σ represents the bias parameter; σ represents the sigmoid activation function.

[0070] S7. During the model training phase, the loss function L consists of three parts, namely L for the main entity recognition module. subject L of the object entity recognition module object L and the subject-object pair relationship classification module rc The loss function formulas for each module are as follows:

[0071]

[0072]

[0073]

[0074] in, and These represent the target values ​​of each element in the subject entity table, object entity table, and subject-object pair relationship classification table, respectively. and Let each element in each table represent its predicted value. In each subtask, the loss function used is BCELoss, and the training objective is to minimize the sum of the three losses:

[0075] L = L subject +L object +L rc (16)

[0076] S8. During the inference phase, we combine the results from the subject entity table, object entity table, and subject-object pair relation classification table to extract relation triples. For each predicted standard triple... Where p represents a predefined relation category label; c and c' represent the labels for the subject entity and object entity types; subscripts i,j and m,n represent the starting and ending indices of the subject entity s and object entity o, respectively; they should satisfy the following conditions:

[0077]

[0078] in, The score for entity type c represents the complete span from the start position i to the end position j of the main entity. The score for entity type c' represents the complete span from the start position m to the end position n of the object entity. The score for a relation of type l represents the connection between the starting position i of the subject entity and the ending position m of the object entity; λ s , λ o and λr These represent the threshold values ​​for each element in the subject entity recognition, object entity recognition, and subject-object pair relationship classification modules, respectively. The threshold values ​​for all three sub-tasks can be set to 0.5.

[0079] The method proposed in this embodiment divides the task into three interactive sub-tasks: subject entity recognition, object entity recognition, and subject-object pair classification. Furthermore, it utilizes a dual affine mechanism and a shared CNN module to achieve bidirectional interaction between the three sub-tasks. Experimental verification was conducted in other embodiments, and the results show that the proposed method achieves good results on the NYT and WEBNLG public datasets. Ablation experiments demonstrate the necessity of further task decomposition and the importance of bidirectional interaction between sub-tasks.

[0080] Example 2:

[0081] This embodiment provides an entity relation joint extraction system based on an interactive dual affine mechanism, including:

[0082] The data acquisition module is configured to acquire text data.

[0083] The tokenization module is configured to tokenize the acquired text data.

[0084] The task decomposition module is configured to decompose the task of jointly extracting entity relations from text data into three joint learning sub-tasks: subject entity recognition, object entity recognition, and subject-object pair relation classification.

[0085] The mapping module is configured to map the tokenized text data to the subject entity table, the object entity table, and the subject-object pair relationship classification table based on a preset dual affine mechanism model.

[0086] The extraction module is configured to extract entity relationships from the jointly obtained subject entity table, object entity table, and subject-object pair relationship classification table.

[0087] The working method of the system is the same as the entity relation joint extraction method based on interactive dual affine mechanism in Embodiment 1, and will not be repeated here.

[0088] Example 3:

[0089] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the entity relation joint extraction method based on an interactive dual affine mechanism described in Embodiment 1.

[0090] Example 4:

[0091] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the entity relation joint extraction method based on the interactive dual affine mechanism described in Embodiment 1.

[0092] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. A method for joint extraction of entity relations based on an interactive dual affine mechanism, characterized in that, include: Get text data; Tokenize the acquired text data; The task of jointly extracting entity relations from text data is decomposed into three joint learning sub-tasks: subject entity recognition, object entity recognition, and subject-object pair relation classification. Based on the pre-defined dual affine mechanism model, the tokenized text data is mapped to the subject entity table, the object entity table, and the subject-object pair relationship classification table; The tokenized text data is encoded to obtain a hidden vector representation of each word in the sentence with contextual information. The encoded hidden vector representations are transformed to obtain a hidden vector for each token as the subject representation and a hidden vector for the main representation. The hidden vectors of the obtained subject representations are encoded to obtain a hidden vector for the start position and a hidden vector for the end position of each subject representation. Using a double affine mechanism model, the hidden vectors for the start position and the hidden vectors for the end position of all obtained subject entities are mapped to obtain the subject entity table. Similarly, the object entity table is obtained. In a convolutional neural network with shared parameters, the contextual information representation of sentence span is extracted, and the span information is fused into the representation of a single token to obtain the subject vector representation and the object vector representation. Using a dual affine mechanism model, the obtained subject vector representation and object vector representation with span information are mapped to obtain a subject-object pair relationship classification table. The entity relationships are extracted from the combined subject entity table, object entity table, and subject-object pair relationship classification table.

2. The entity relation joint extraction method based on interactive dual affine mechanism as described in claim 1, characterized in that, The acquired text data is tokenized using a pre-trained language model and processed into token form. During model training, for each line of data in the training set, based on the given subject entity, subject-object pair relation, and object entity triple form, each subject entity is first processed into token form. Then, based on the subject entity token, the start index and end index of the entity in the sentence token are found as the span of the subject entity in the sentence.

3. The entity relation joint extraction method based on interactive dual affine mechanism as described in claim 2, characterized in that, The encoded hidden vector representation is transformed to obtain the hidden vector of each token as the object representation and the hidden vector of the object representation; The hidden vectors of the obtained object representations are encoded to obtain the hidden vectors of the start and end positions of each object representation as the object entity. Using a dual affine mechanism model, the hidden vectors of the start and end positions of all obtained object entities are mapped to obtain an object entity table.

4. The entity relation joint extraction method based on interactive dual affine mechanism as described in claim 1, characterized in that, Each element in the subject entity table, object entity table, and subject-object pair relationship classification table is compared with a preset threshold to obtain the subject entities, object entities, and subject-object pairs relationships that meet the conditions.

5. The entity relation joint extraction method based on interactive dual affine mechanism as described in claim 1, characterized in that, The loss functions include the subject entity recognition loss function, the object entity recognition loss function, and the subject-object pair relationship classification loss function; during model training, the goal is to minimize the sum of the subject entity recognition loss function, the object entity recognition loss function, and the subject-object pair relationship classification loss function.

6. A joint entity relation extraction system based on an interactive dual affine mechanism, characterized in that, include: The data acquisition module is configured to acquire text data. The tokenization module is configured to tokenize the acquired text data. The task decomposition module is configured to decompose the task of jointly extracting entity relations from text data into three joint learning sub-tasks: subject entity recognition, object entity recognition, and subject-object pair relation classification. The tokenized text data is encoded to obtain a hidden vector representation of each word in the sentence with contextual information. The encoded hidden vector representations are transformed to obtain a hidden vector for each token as the subject representation and a hidden vector for the main representation. The hidden vectors of the obtained subject representations are encoded to obtain a hidden vector for the start position and a hidden vector for the end position of each subject representation. Using a double affine mechanism model, the hidden vectors for the start position and the hidden vectors for the end position of all obtained subject entities are mapped to obtain the subject entity table. Similarly, the object entity table is obtained. In a convolutional neural network with shared parameters, the contextual information representation of sentence span is extracted, and the span information is fused into the representation of a single token to obtain the subject vector representation and the object vector representation. Using a dual affine mechanism model, the obtained subject vector representation and object vector representation with span information are mapped to obtain a subject-object pair relationship classification table. The mapping module is configured to map the tokenized text data to the subject entity table, the object entity table, and the subject-object pair relationship classification table based on a preset dual affine mechanism model. The extraction module is configured to extract entity relationships from the jointly obtained subject entity table, object entity table, and subject-object pair relationship classification table.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the entity relation joint extraction method based on the interactive dual affine mechanism as described in any one of claims 1-5.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the entity relation joint extraction method based on the interactive dual affine mechanism as described in any one of claims 1-5.

Images