Heterogeneous graph neural network-based entity relationship joint extraction model and construction method

Abstract

The invention relates to the field of natural language processing, and provides an entity relation joint extraction model based on a heterogeneous graph neural network and a construction method. A conditional attention mechanism is adopted to introduce predefined relationship type information in an original text sequence as priori knowledge into a dynamic feature word vector of the original text sequence fused with context semantic information on word granularity to obtain a feature word vector fused with the context semantic information and a relationship type; when the heterogeneous graph is constructed based on the entity types, virtual relation edges are added between the entity pairs existing in the heterogeneous graph, the relation between the two connected entities is expressed through the virtual relation edges, and combined modeling of the entities and the relation is achieved; context dependency between entities and relationships and relationship interaction among multiple entities can be better captured through joint modeling, the complexity of actual relationships can be better restored through joint modeling of the relationships, and the consistency and accuracy of extracting the overlapping relationships among the entities are more efficiently improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to an entity relationship joint extraction model based on a heterogeneous graph neural network and a construction method. Background Art

[0002] With the vigorous development of information technology, information resources on the Internet are showing an explosive growth trend. In the case of large-scale and rapidly growing text information, how to efficiently extract, organize and utilize the knowledge contained in massive information resources has become a new challenge. In this context, knowledge graphs, as a large-scale knowledge storage structure, have been proposed and studied in depth and extensively. Knowledge graphs use entities and relationships as the basic elements of knowledge expression, and entities and relationships mainly come from text data, especially large-scale and wide-ranging unstructured natural language text data. Therefore, information extraction technologies such as named entity recognition (NER) and relation extraction (RE) that mine entities and relationships from massive natural language texts have emerged and become a research hotspot for knowledge graph construction.

[0003] Early entity relationship extraction adopted a pipeline approach, and many scholars regarded named entity recognition (NER) and relation extraction (RE) as two independent subtasks in the information extraction task. They conducted research by training the two subtasks separately in the entity relationship extraction process and sharing basic semantic connection information. However, this approach has some potential problems such as error propagation, that is, when there are entity labeling errors in the NER task, these errors may affect the performance of the relation extraction task.

[0004] In view of the many drawbacks of the pipeline model, the entity relationship extraction method based on joint learning has gradually entered the field of vision of experts and scholars. Joint learning is also called end-to-end extraction method. This method can achieve the integration of entity recognition and relationship extraction. In recent years, various entity relationship extraction models have been proposed, but most of them only improve the accuracy of normal entity relationship extraction. There is still no good model framework for difficult tasks such as entity nesting and overlapping relationship extraction in entity relationship extraction tasks. Summary of the invention

[0005] In view of the above problems, the present invention is proposed to provide an entity relationship joint extraction model and construction method based on heterogeneous graph neural networks that overcome the above problems or at least partially solve the above problems.

[0006] One aspect of the present invention provides a method for constructing an entity relationship joint extraction model based on a heterogeneous graph neural network, the method comprising: The original text sequence is input into the ERNIE3.0 pre-training model for pre-training to obtain the first dynamic feature word vector of the original text sequence that integrates contextual semantic information at the word granularity; A conditional attention mechanism is used to introduce the predefined relationship type in the original text sequence as prior knowledge into the first dynamic feature word vector, thereby obtaining a second feature word vector that integrates contextual semantic information and relationship type information; Input the second feature word vector into a preset BILSTM model for encoding operation to obtain a forward hidden state sequence and a reverse hidden state sequence, merge the forward hidden state sequence and the reverse hidden state sequence to obtain a bidirectional encoding representation of the second feature word vector, and generate a third feature word vector according to the bidirectional encoding representation of the second feature word vector; The third feature word vector is converted into a preset table output, and a multi-head selection method is used to perform entity annotation on the entity position and entity type in the original text sequence in the table output; According to the table output after entity annotation, a heterogeneous graph is constructed with different entity types as different types of nodes, wherein the initial feature vector of each node includes the entity position information and the fused feature vector of the third feature word vector, and virtual relationship edges are added between entity pairs existing in the heterogeneous graph; A corresponding heterogeneous graph convolution layer is set for each relationship type, and the corresponding heterogeneous graph convolution operation is performed on each relationship type in the heterogeneous graph convolution training to generate the corresponding edge prediction results to obtain the triple prediction output of each relationship type. The triplet specifically includes the head entity representation, the relationship type and the tail entity representation.

[0007] Optionally, using a conditional attention mechanism to introduce a predefined relationship type in the original text sequence as prior knowledge into the first dynamic feature word vector includes: Project the first dynamic feature word vector to the value, key, and query vector spaces respectively through a linear layer; Mapping each predefined relationship type into a relationship embedding vector with the same dimension as the query vector through an embedding layer, and adding the relationship embedding vector to the query vector to obtain a query vector that integrates the relationship type information; Split the query vectors of value, key, and fused relation type information into multiple heads respectively; Calculate the dot product of the query vector and the key vector that incorporate the relationship type information to obtain the attention score matrix energy; Applying the softmax function to normalize the attention score matrix to obtain the attention weight, using the attention weight to perform weighted summation on the value vector, and using the obtained attention output as the second feature word vector that integrates contextual semantic information and relationship information. Using the adaptive multi-head attention mechanism to introduce relationship information as prior knowledge into the first dynamic feature word vector includes:

[0008] Map each predefined relationship type into an embedding vector of fixed dimension; Adopt adaptive multi-head attention mechanism linear transformation to calculate the attention score of each head and normalize it through softmax function; The attention weight of each relation type is calculated using the dot product between the query and the key, and the attention weight of each relation type is multiplied by the attention matrix to adjust the distribution of the attention weight; The value vector is weighted and summed using the adjusted attention weights to obtain the second feature word vector that combines contextual semantic information and relationship information.

[0009] Optionally, forming a third feature word vector according to the bidirectional encoding representation of the second feature word vector includes: The bidirectional encoding representation of the second feature word vector is feature fused through a fully connected layer to form a third feature word vector.

[0010] Optionally, the third feature word vector is converted into a preset table output, including: The output of the tensor shape (batch batch_size, sequence length seq_len, hidden dimension hidden_dim) of the third feature word vector is expanded to seq_len size by adding a new dimension at the third position, and an intermediate tensor output in the form of (batch_size, seq_len, seq_len, hidden_dim) is generated to obtain the first hidden state combination matrix of the entity start position at each word granularity; The obtained intermediate tensor output is transposed and the positions of the first dimension and the second dimension are exchanged to obtain the second hidden state combination matrix of the entity end position at each word granularity; The first hidden state combination matrix and the second hidden state combination matrix are concatenated on the last dimension representing the feature to obtain a tabular output.

[0011] Optionally, a multi-head selection method is used to label entity positions and entity types in the original text sequence in the tabular output, including: The concatenated hidden state combination output in table form is mapped to the preset label space through the fully connected layer, and the softmax activation function is used to map the concatenated hidden state combination to the final label probability to obtain the entity position and entity type information in the original text sequence.

[0012] Optionally, the method further comprises: A multi-head attention mechanism is used to fuse the entity position information of each entity and the third feature word vector, and the obtained fused feature vector is used as the initial feature vector of the node corresponding to the current entity; Construct a dictionary form for each entity node in the heterogeneous graph, where the key of the dictionary is the type of entity and the value of the dictionary is a set of initial feature vectors of all entities under the entity type.

[0013] Optionally, performing a corresponding heterogeneous graph convolution operation on each relationship type in the heterogeneous graph convolution training to generate a corresponding edge prediction result to obtain a triple prediction output of each relationship type includes: For each relationship type, the heterogeneous graph and the initial feature vectors of all entity nodes in the heterogeneous graph are used as input data, and the heterogeneous graph convolution operation corresponding to the current relationship type is used to process the input data to obtain the triple prediction result of the current relationship type.

[0014] Optionally, the method further comprises: In heterogeneous graph convolution training, through the process of message passing on the graph, the initial feature vectors of each entity node's adjacent nodes are aggregated to obtain the context information of the current node and perform mean aggregation to obtain a new feature representation. The new feature representation after mean aggregation is fused with the initial feature vector of the current node, and the feature representation of the current node is updated according to the fused features.

[0015] Another aspect of the present invention further provides an entity relationship joint extraction model based on a heterogeneous graph neural network, the model includes a named entity recognition unit and a relationship classification unit, the named entity recognition unit includes a pre-training module, a relationship information fusion module, a global feature recognition module and an entity labeling module, the relationship classification unit includes a heterogeneous graph construction module and a relationship classification module, wherein: A pre-training module is used to input the original text sequence into the ERNIE3.0 pre-training model for pre-training, and obtain the first dynamic feature word vector of the original text sequence that integrates contextual semantic information at the word granularity; A relation information fusion module, used to introduce the predefined relation type in the original text sequence as prior knowledge into the first dynamic feature word vector by using a conditional attention mechanism, so as to obtain a second feature word vector that integrates contextual semantic information and relation type information; A global feature recognition module, used for inputting the second feature word vector into a preset BILSTM model for encoding operation to obtain a forward hidden state sequence and a reverse hidden state sequence, merging the forward hidden state sequence and the reverse hidden state sequence to obtain a bidirectional encoding representation of the second feature word vector, and generating a third feature word vector according to the bidirectional encoding representation of the second feature word vector; An entity labeling module, used for converting the third feature word vector into a preset table output, and using a multi-head selection method to label entity positions and entity types in the original text sequence in the table output; A heterogeneous graph construction module is used to construct a heterogeneous graph with different entity types as different types of nodes according to the table output after entity annotation, wherein the initial feature vector of each node includes the entity position information and the fused feature vector of the third feature word vector, and virtual relationship edges are added between entity pairs existing in the heterogeneous graph; The relationship classification module is used to set the corresponding heterogeneous graph convolution layer for each relationship type, and perform the corresponding heterogeneous graph convolution operation on each relationship type in the heterogeneous graph convolution training, generate the corresponding edge prediction results to obtain the triple prediction output of each relationship type, and the triple specifically includes the head entity representation, the relationship type and the tail entity representation.

[0016] The third aspect of the present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for constructing the above-mentioned entity relationship joint extraction model based on heterogeneous graph neural network are implemented.

[0017] The entity relationship joint extraction model and construction method based on heterogeneous graph neural network provided by the embodiment of the present invention introduces relationship information as prior knowledge into the dynamic feature word vector of the original text sequence that fuses contextual semantic information at the word granularity by adopting an adaptive multi-head attention mechanism, thereby obtaining a feature word vector that fuses contextual semantic information and relationship information. Moreover, when constructing a heterogeneous graph based on entity types, virtual relationship edges are added between entity pairs in the heterogeneous graph to facilitate heterogeneous graph convolution operations, thereby realizing joint modeling of entities and relationships. Through joint modeling, the contextual dependency between entities and relationships and the relationship interaction between multiple entities can be better captured. Through joint modeling, these relationships can also better restore the complexity of actual relationships, and more efficiently improve the consistency and accuracy of extracting overlapping relationships between entities. Furthermore, the present invention adopts a multi-headed annotated named entity recognition method, which can solve the difficult problems such as entity nesting and overlapping relationships in the Chinese entity relationship joint extraction task at one time, and can more accurately and efficiently identify entity relationship triplets in unstructured text sequences, ensuring that all important information in complex texts can be more fully extracted.

[0018] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0020] Figure 1 A flowchart of a method for constructing an entity relationship joint extraction model based on a heterogeneous graph neural network provided in an embodiment of the present invention; Figure 2 A schematic diagram of a tabular output formed in a method for constructing an entity relationship joint extraction model based on a heterogeneous graph neural network proposed in an embodiment of the present invention; Figure 3 A structural block diagram of an entity relationship joint extraction model based on a heterogeneous graph neural network proposed in an embodiment of the present invention; Figure 4 This is a diagram of the overall architecture of the entity relationship joint extraction model based on heterogeneous graph neural network proposed in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0022] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0023] Those skilled in the art will understand that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined.

[0024] The present invention is aimed at the entity relationship extraction task in the field of natural language processing. Aiming at the difficulties of extracting entity nesting and overlapping relationships in the current entity relationship joint extraction model, a method for constructing an entity relationship joint extraction model based on a heterogeneous graph neural network is proposed. Figure 1 The flowchart of the method for constructing an entity relationship joint extraction model based on a heterogeneous graph neural network according to an embodiment of the present invention is schematically shown. Figure 1 The method for constructing an entity relationship joint extraction model based on a heterogeneous graph neural network in an embodiment of the present invention specifically includes the following steps:

[0025] S11. Input the original text sequence into the ERNIE3.0 pre-training model for pre-training to obtain the first dynamic feature word vector of the original text sequence that integrates contextual semantic information at the word granularity.

[0026] In this embodiment, the original text sequence, such as {x1, x2, x3, …, x255, x256}, is first input into the ERNIE3.0 pre-training model for pre-training to obtain the first dynamic feature word vector of the text sequence that integrates contextual semantic information at the word granularity, and the tensor shape is (batch, seq-len, hidden-size). Among them, the first dimension represents the number of samples in a batch (batch), the second dimension represents the sequence length, and the third dimension is the hidden layer dimension. The hidden dimension is the built-in dimension of the ERNIE3.0 pre-training model, which is 768.

[0027] Among them, the ERNIE pre-training model is a pre-training model released by Baidu, including different versions such as ERNIE1.0, 2.0, 3.0 and 3.5. The present invention uses the latest version of the ERNIE3.0 pre-training model to pre-train sample sentences to vectorize the text where the entity is located, wherein the text is vectorized, that is, to express these texts in the form of data. Specifically, it is in the form of vectors, so it is also called word vectors, and the pre-training model will assign different weights to the words in the sentence according to the order of sentences and tasks each time to form different dynamic word vectors. The ERNIE3.0 pre-training model is different from the BERT pre-training model in that the former uses a variety of joint training tasks, which can better understand the semantics and structural information of the text, and uses technical means such as multi-head attention mechanisms to enhance the modeling ability of the model. Therefore, ERNIE3.0 can better process natural language text data, improve the performance and robustness of the model, and can be more suitable for Chinese language tasks.

[0028] S12. Use the conditional attention mechanism to introduce the predefined relationship types in the original text sequence as prior knowledge into the first dynamic feature word vector, and obtain a second feature word vector that integrates contextual semantic information and relationship type information.

[0029] Specifically, the predefined relationship types in the original text sequence specifically refer to a set of relationship categories that are predefined before model training and testing. These relationship types are usually selected based on specific task requirements and application scenarios, and are clearly labeled in the dataset. These predefined relationship types provide a clear goal for the model to identify and classify the relationship between entities in the text. For example: "Suzhou XXX Technology Co., Ltd. was registered and established on November 4, 2011 at the Suzhou XX District Market Supervision and Administration Bureau" The predefined relationship is "Date of Establishment".

[0030] S13. Input the second feature word vector into a preset BILSTM model for encoding operation to obtain a forward hidden state sequence and a reverse hidden state sequence, merge the forward hidden state sequence and the reverse hidden state sequence to obtain a bidirectional encoding representation of the second feature word vector, and generate a third feature word vector based on the bidirectional encoding representation of the second feature word vector.

[0031] Specifically, forming the third feature word vector according to the bidirectional encoding representation of the second feature word vector in step S13 includes: performing feature fusion on the bidirectional encoding representation of the second feature word vector through a fully connected layer to form the third feature word vector.

[0032] In this embodiment, the second feature word vector that integrates text semantic information and relationship type information is input into the BILSTM model for further encoding operations. BiLSTM encodes the text sequence in two directions to obtain forward and reverse hidden state sequences respectively. The hidden state sequences obtained by forward and reverse encoding are merged together to obtain a bidirectional encoding representation of the entire sequence. In this way, the model can make full use of the forward and reverse information in the text sequence, so as to better capture the long dependencies in the sequence, so that the model can fully capture the contextual information. The output tensor in the form of (batch_size, seq_len, 2* hidden_dim) is restored to the form of (batch_size, seq_len, hidden_dim) through the fully connected layer to obtain the third feature word vector of the entire text sequence.

[0033] BiLSTM is a model based on RNN, which is good at capturing dependencies in sequence data. The present invention can further enhance the temporal dependencies and local context information in the sequence by using BiLSTM on the basis of ERNIE3.0 encoding vector. Although ERNIE3.0 can capture long-range dependencies well, sometimes the capture of local temporal dependencies may not be as detailed as RNN-type models. When processing time series data, BiLSTM is good at capturing local temporal dependencies. Combining it with the global features of ERNIE3.0 can more comprehensively capture various dependencies of sentences. This enables the model to better understand the overall meaning and complex semantic relationships of the sentence.

[0034] S14, converting the third feature word vector into a preset table output, and using a multi-head selection method to perform entity annotation on the entity position and entity type in the original text sequence in the table output.

[0035] Among them, entity categories may include place name entity (Loc), organization entity (Org), name entity (Name), position entity (Pos) and time entity (Time).

[0036] In a specific example, the original text sequence is part of a sample sentence, and the sample sentence is as follows: '{"text": "Suzhou XXX Technology Co., Ltd. was registered and established in Suzhou XX District Market Supervision Administration on November 4, 2011",' \ ' "spo_list": [{"predicate": "Date of Establishment", "object_type": {"@value": "Date"}, "subject_type": "Organization", "object": {"@value": "November 4, 2011"}, "subject": "Suzhou XXX Technology Co., Ltd."}]}' Specifically, the step S14 of converting the third feature word vector into a preset table output includes: adding a new dimension to the output of the tensor shape (batch batch_size, sequence length seq_len, hidden dimension hidden_dim) of the third feature word vector at the third position and expanding it to the size of seq_len, generating an intermediate tensor output in the form of (batch_size, seq_len, seq_len, hidden_dim) to obtain a first hidden state combination matrix of the entity start position at each word granularity; transposing the obtained intermediate tensor output and exchanging the positions of the first dimension and the second dimension to obtain a second hidden state combination matrix of the entity end position at each word granularity; and concatenating the first hidden state combination matrix and the second hidden state combination matrix on the last dimension representing the feature to obtain a table output.

[0037] In this embodiment, the output third feature word vector is converted into a table mode for subsequent multi-head annotated entity prediction and loss calculation. The principle is to copy the tokens with the number of seq_len for seq_len times. Then add them to the corresponding positions of its transposed matrix. The specific implementation is to add a new dimension at the third position to the output in the form of (batch_size, seq_len, hidden_dim) and expand it to the size of seq_len, becoming (batch_size, seq_len, seq_len, hidden_dim) to obtain the hidden state combination of the start position of the entity at each word granularity. Then transpose the above tensor and exchange the positions of the first dimension and the second dimension to obtain the hidden state combination of the end position of the entity at each word granularity. Finally, concatenate the two matrices on the last dimension representing the features to obtain the output in the form of a table. The present invention can convert the output of the model into the form of a table for output. One of the two matrices is to change the dimension of the output vector of the model into the hidden state containing the start position of the entity, and the other matrix is the hidden state containing the end position of the entity formed by transposing the first matrix and exchanging the second and third dimension vectors. The specific concatenation method is to use the cat function in the fourth dimension to concatenate them together according to the specified fourth dimension, that is, concatenate the information of the start position and the end position in the fourth dimension to obtain a tensor of (batch_size, seq_len, seq_len, hidden_size * 2). Generate a matrix containing the start and end positions of the entities in the text sequence and further convert it into the output in the form of a table.

[0038] Specifically, the multi-head selection method in step S14 is used to perform entity annotation on the entity positions and entity types in the original text sequence in the table-form output, including: mapping the concatenated hidden state combination of the table-form output to a preset label space through a fully connected layer, and using the softmax activation function to map the concatenated hidden state combination to the final label probability to obtain the entity position and entity type information in the original text sequence.

[0039] Among them, the label space is predefined and includes the type dictionary of entities. Specifically, as shown in the annotation matrix Figure 2 The cross coordinate of the characters "Li" and "Ming", that is, the position [1, 2] is 2, indicating that the start position of the entity in the text sequence is 1 and the end position is 2, and the entity type represented is 2. It can be known from the predefined entity type dictionary that the entity type corresponding to 2 is "person name".

[0040] In this embodiment, the hidden state output in the form of a table is mapped to the preset label space through the fully connected layer. Since there are multiple entity types and different numerical correspondences are presented in the table, the softmax activation function is used to map the hidden state combination to the final label probability, and finally the entity position and entity type information in the sentence are obtained. That is, the entity position and entity type information in the original text sequence are predicted; the entity position and entity type information in the original text are predicted by the model and expressed in the form of a table.

[0041] S15. A heterogeneous graph is constructed based on the table output after entity annotation, with different entity types as different types of nodes. The initial feature vector of each node includes the entity location information and a fused feature vector of the third feature word vector, and virtual relationship edges are added between entity pairs in the heterogeneous graph.

[0042] Specifically, when constructing a heterogeneous graph, the method also includes: using a multi-head attention mechanism to fuse the entity position information of each entity and the third feature word vector, and using the obtained fused feature vector as the initial feature vector of the node corresponding to the current entity; constructing a dictionary form of each entity node in the heterogeneous graph, the key of the dictionary is the type of entity, and the value of the dictionary is a set of initial feature vectors of all entities under the entity type.

[0043] Specifically, constructing a dictionary form of entity nodes can provide initial feature vector representations for different types of entity nodes when constructing heterogeneous graphs. For example, each type of entity represents a different node type, but there are also differences between entity nodes of the same type. By setting the dictionary, an independent initial feature vector can be set for each node.

[0044] In this embodiment, a heterogeneous graph is constructed based on the output information of the entity recognition submodule. Different entity types are used as different types of nodes. When designing the initial feature vector of the node, it is not selected to directly generate the initial node feature vector by embedding the entity, but to use a multi-head attention mechanism to fuse the position feature and the third feature word vector in the text entity recognition submodule to obtain the initial feature vector of the node, that is, the entity. The initial feature of the customized node can enhance the expression ability of the model. By integrating the feature information of the entity position information and the global feature of the text context, the model can be helped to better capture the key features and patterns in the data, thereby improving the performance and generalization ability of the model and accelerating the convergence speed and optimization effect of the model. The dictionary form of the entity node is constructed, the key of the dictionary is the type of entity, and the value of the dictionary is a list containing all entity feature vectors under the entity type.

[0045] In this embodiment, when constructing a heterogeneous graph, only the type of entity and the corresponding feature information are known, but the clear relationship type between entities is unknown. Therefore, the present invention uses virtual relationship edges to connect entities. Virtual relationship edges can be used as an initialization method to provide an initial connection structure so that the model can learn the potential relationship between entities during the training process. By adding virtual relationship edges, entity nodes can better exchange and merge information. In this way, even if there is no clear relationship information in the initial stage, nodes can still transfer features and update representations through virtual edges. These edges can be regarded as the initial channels for the model to gradually give actual semantics during the learning process. The present invention adds virtual relationship edges between all possible entity pairs to facilitate subsequent heterogeneous graph convolution operations and edge type prediction. When there is no clear relationship edge in the input data, the virtual relationship edge can increase the expressive power of the graph model, enabling it to capture richer relationships between nodes. By introducing virtual relationship edges during the training process, the model can learn more general node-to-node relationship patterns, thereby improving the prediction ability of unknown node relationships. When the types of the source node and the target node are the same, only the edges of adjacent nodes are added to avoid adding duplicate edges.

[0046] S16. Setting a corresponding heterogeneous graph convolution layer for each relationship type, and performing a corresponding heterogeneous graph convolution operation on each relationship type in the heterogeneous graph convolution training, generating a corresponding edge prediction result to obtain a triple prediction output for each relationship type. The triple specifically includes a head entity representation, a relationship type, and a tail entity representation.

[0047] The method for constructing an entity relationship joint extraction model based on a heterogeneous graph neural network provided by an embodiment of the present invention introduces relationship type information as prior knowledge into the dynamic feature word vector of the original text sequence that fuses contextual semantic information at the word granularity by adopting a multi-head attention mechanism, thereby obtaining a feature word vector that fuses contextual semantic information and relationship information. Moreover, when constructing a heterogeneous graph based on entity type, virtual relationship edges are added between entity pairs existing in the heterogeneous graph, and the relationship between two connected entities is represented by the virtual relationship edges, thereby realizing joint modeling of entities and relationships. The joint modeling can better capture the contextual dependency between entities and relationships and the relationship interaction between multiple entities. The joint modeling can also better restore the complexity of the actual relationships and more efficiently improve the consistency and accuracy of extracting overlapping relationships between entities.

[0048] Entity-relationship joint extraction automatically identifies entities, entity types, and specific relationship types between entities by modeling text information, and finally predicts and outputs them in the form of {head entity representation, relationship type, tail entity representation}. In recent years, various entity relationship extraction models have been proposed, but most of them only improve the accuracy of normal entity relationship extraction. There is still no good model framework for difficult tasks such as entity nesting and overlapping relationship extraction in entity relationship extraction tasks.

[0049] Entity nesting problem: In real life, there is a situation where one entity is nested in another entity. The same word may belong to different entities. For example, "Tianjin University" is a university, and "Tianjin" is also a place. The joint extraction model needs to incorporate richer contextual semantic information to identify the current entity type. The situation of a large number of entity nesting increases the difficulty of joint extraction. As for the relationship overlap problem, different relationship types can exist in the same sentence, and there can be multiple relationship types between the same entities. Different relationships may also contain some hidden information. For example, "Beijing is the capital of China", there are two relationship types between "China" and "Beijing": "located in" and "capital".

[0050] In order to enhance the system's ability to recognize entity boundary information, the present invention uses the Multi-Head selection method to define the recognition task of entities and nested entities as a multi-head selection problem of entity boundaries. The main principles are as follows: the system is required to output a tabular output, that is, a matrix, which contains the annotation information of the entity. Specifically, if an entity exists in the text, the corresponding entity type value will be output in the output matrix at the position corresponding to the start and end tokens of the entity; if the entity does not exist in the text, the output is 0. In addition, since it is impossible for the tail position index of the entity to be less than the head position index of the entity, all elements in the lower triangle of the matrix are set to -1 to improve the generalization performance of the model, reduce the impact of noise and inaccurate information learned by the model in the training data, and make it more focused on real data patterns and features, such as Figure 2 shown.

[0051] In this embodiment, entity types have been predefined and converted to each other in the form of dictionaries. Figure 2 This is just an example. The actual entity type dictionary is { "UNENTITY": 0, "Character": 1, "Date": 2, "Institution": 3, "Text": 4, "language": 5, "Country": 6, "Film and TV Works": 7, "Awards": 8, "Entertainment Figures": 9, "Music Album": 10, "Songs": 11, "Number": 12, "Location": 13, "Book Works": 14, "school": 15, "Enterprise": 16, "Historical Figures": 17, "TV Variety Show": 18, "Company / Brand": 19, "Works": 20, "Literary Works": 21, "City": 22, "Attractions": 23, "Climate": 24, "Administrative District": 25, Subject Major: 26 } The method for constructing an entity relationship joint extraction model based on a heterogeneous graph neural network provided in an embodiment of the present invention adopts a multi-head labeled named entity recognition method, which can solve the difficult problems such as entity nesting and overlapping relationships in the Chinese entity relationship joint extraction task at one time, and can more accurately and efficiently identify entity relationship triplets in unstructured text sequences, ensuring that all important information in complex texts can be more fully extracted.

[0052] In the embodiment of the present invention, the use of the conditional attention mechanism in step S12 to introduce the predefined relationship type in the original text sequence as prior knowledge into the first dynamic feature word vector includes the following steps not shown in the figure: S121, project the first dynamic feature word vector to the values, keys, and queries vector spaces respectively through a linear layer: in, is a value vector, is the key vector, is the query vector; , , is the weight matrix of linear projection, , , The weight matrices used to convert the input embedding vectors into value vectors, key vectors, and query vectors are trainable parameters in the model.

[0053] S122: Map each predefined relationship type into a relationship embedding vector with the same dimension as the query vector through an embedding layer, and add the relationship embedding vector to the query vector to obtain a query vector that integrates the relationship type information.

[0054] S123, splitting the value vector, the key vector, and the query vector fused with the relationship type information into multiple headers respectively; Specifically, the values, keys, and queries containing relationship information are split into multiple heads. On the one hand, the noise and error that may be caused by single-head attention can be reduced through the joint action of multiple heads, thereby enhancing the robustness of the model. On the other hand, the multi-head attention mechanism enables each word to be interactively calculated in multiple subspaces, which can more comprehensively capture the dependency relationship between words and improve the performance and effect of the model.

[0055] (batch, sequence length, number of split heads, dimension of each head). In the multi-head attention mechanism, head_dim (the dimension of each head) is determined by the embedding size (embed_size) and the number of heads (heads). Specifically, head_dim is obtained by dividing embed_size into heads, and the dimension of each head should satisfy:

[0056] S124. Calculate the dot product of the query vector and the key vector that integrate the relationship type information to obtain the attention score matrix energy.

[0057] Specifically, we then calculate the dot product of the query vector and the key vector that incorporate the relational information to obtain the attention score matrix energy, which is then scaled. At the same time, we use a mask to ignore the padding positions in the text sequence.

[0058] S124. Apply the softmax function to normalize the attention score matrix to obtain the attention weight attention, use the attention weight to perform weighted summation on the value vector, and use the obtained attention output as the second feature word vector that integrates the contextual semantic information and relationship information.

[0059] Specifically, softmax is applied to the attention score matrix to obtain the attention weight attention. The value vector is weighted summed using the attention weight to obtain the attention output out.

[0060] Finally, attention_output contains the text representation vector that incorporates the relation type information. When the relation type embedding vector is combined with the query vector to calculate the attention score, the attention score reflects the importance of the relation type to each word. The attention score is normalized by the softmax operation, making the weight of the relation information adaptive overall.

[0061] The present invention can effectively integrate relationship type information into sentence text vectors through the conditional self-attention mechanism, and adaptively adjust relationship weights, thereby improving the performance of entity relationship joint extraction tasks.

[0062] In this embodiment, in order to avoid the entity redundancy caused by the extracted entities not being able to form a valid entity relationship triple in the end, the present invention uses an adaptive multi-head attention mechanism (AMHAT) to introduce relationship information as prior knowledge into the first dynamic feature word vector of the text, so that the relationship information that may exist between entities will be considered before extracting the entity. The specific method is: first embed all predefined relationship types, map each relationship type to an embedding vector of a fixed dimension, and then calculate the attention score of each head through a multi-head attention mechanism linear transformation, and normalize it through a softmax function to meet the requirements of the attention mechanism. After that, the attention weight is calculated using the dot product of the query and the key, and then the weights of these relationship types are multiplied by the attention matrix to adjust the distribution of attention, that is, to increase or decrease the importance of different relationship types in attention. Finally, the value vector is weighted and summed using the adjusted attention weight to obtain the final output, and the second feature word vector that integrates the adaptively adjusted predefined relationship type information is obtained, and the tensor shape is [batch, seq-len, hidden-size].

[0063] In an embodiment of the present invention, in step S16, a corresponding heterogeneous graph convolution operation is performed on each relationship type in the heterogeneous graph convolution training to generate a corresponding edge prediction result to obtain a triple prediction output for each relationship type, including: for each relationship type, the initial feature vectors of the heterogeneous graph and all entity nodes in the heterogeneous graph are used as input data, and the input data is processed using the heterogeneous graph convolution operation corresponding to the current relationship type to obtain a triple prediction result for the current relationship type.

[0064] Furthermore, the method also includes: in heterogeneous graph convolution training, through the process of message passing on the graph, for each entity node, the initial feature vectors of its adjacent nodes are aggregated to obtain the context information of the current node and perform mean aggregation to obtain a new feature representation, and the new feature representation after mean aggregation is fused with the initial feature vector of the current node, and the feature representation of the current node is updated according to the fused features.

[0065] When creating a heterogeneous graph convolution layer, the present invention corresponds to a heterogeneous graph convolution layer for each relationship type, and at the same time double loops through all possible source entity types and target entity types, and creates edges for each pair of entity nodes, and the edge type is (head entity representation src_entity, relationship type rel_type, tail entity representation dst_entity). In heterogeneous graph convolution training, through the process of message passing on the graph, for each entity node, the features of its adjacent nodes are aggregated to obtain the context information of the node and perform mean aggregation to obtain a new feature representation, and then the aggregated features are used to fuse with the original features to update the feature representation of the node. Finally, when performing triple prediction, for each relationship type, the model will perform the corresponding heterogeneous graph convolution operation, generate the corresponding edge prediction result and update the feature representation of the node at the same time. For each relationship type, the model will generate triples of head entity, relationship and tail entity, and collect these triples and return them as the output of the model.

[0066] in: Is the relationship type The set of triples generated when .

[0067] Is the relationship type It is a heterogeneous graph convolution operation that accepts the features of the heterogeneous graph G and entity nodes. As input, the generated relation type is The triplet prediction result when .

[0068] This makes it possible to handle situations where there are multiple overlapping relationships between entities. In the heterogeneous graph convolution process, the information between entities, entities and edges, and edges and edges fully interact, thereby better capturing the semantic information between entities and relationships. Finally, the entity-relationship triples are jointly decoded, the sum of the overall loss and the loss of the entity recognition module is calculated, and the weight coefficient is introduced to adjust the loss of the entity recognition module and the loss of the relationship prediction module to flexibly control the relative proportion of the two losses, thereby achieving the purpose of adjusting the loss weight, and finally the model parameters are updated in reverse.

[0069] in: It is the loss of the named entity recognition unit in the entity recognition process, which is used to measure the performance of the model on the entity recognition subtask.

[0070] It is the loss of the relation classification unit in the triple prediction process, which is used to measure the performance of the model in the entity relation triple prediction task.

[0071] and They are the weight coefficients of the entity recognition module loss and the entity relationship triplet prediction module loss respectively.

[0072] Specifically, joint decoding combines the output results of entity recognition and relationship extraction, and converts the final tensor output logits into the starting and ending positions of the head entity, the starting and ending positions of the tail entity, and the relationship type between entities, such as [(starting position of the head entity, ending position), relationship type label between entities, (starting position of the tail entity, ending position)].

[0073] For the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0074] Figure 3 The structural block diagram of the entity relationship joint extraction model based on heterogeneous graph neural network according to an embodiment of the present invention is schematically shown. Figure 3The entity relationship joint extraction model based on heterogeneous graph neural network in the embodiment of the present invention includes a named entity recognition unit and a relationship classification unit, and uses a joint decoding method to enhance the information interaction between the two units, and finally outputs the result in the form of triples. The named entity recognition unit includes a pre-training module 201, a relationship information fusion module 202, a global feature recognition module 203 and an entity labeling module 204, and the relationship classification unit includes a heterogeneous graph construction module 205 and a relationship classification module 206, wherein:

[0075] A pre-training module 201 is used to input the original text sequence into the ERNIE3.0 pre-training model for pre-training, and obtain a first dynamic feature word vector of the original text sequence that integrates contextual semantic information at the word granularity; The relationship information fusion module 202 is used to introduce the predefined relationship type in the original text sequence as prior knowledge into the first dynamic feature word vector by using a conditional attention mechanism, so as to obtain a second feature word vector that integrates contextual semantic information and relationship type information; The global feature recognition module 203 is used to input the second feature word vector into a preset BILSTM model for encoding operation to obtain a forward hidden state sequence and a reverse hidden state sequence, merge the forward hidden state sequence and the reverse hidden state sequence to obtain a bidirectional encoding representation of the second feature word vector, and generate a third feature word vector according to the bidirectional encoding representation of the second feature word vector; An entity labeling module 204, configured to convert the third feature word vector into a preset table output, and to label entity positions and entity types in the original text sequence in the table output using a multi-head selection method; A heterogeneous graph construction module 205 is used to construct a heterogeneous graph with different entity types as different types of nodes according to the table output after entity annotation, wherein the initial feature vector of each node includes the entity position information and the fused feature vector of the third feature word vector, and to add virtual relationship edges between entity pairs existing in the heterogeneous graph, wherein the virtual relationship edges represent the relationship between two connected entities; The relationship classification module 206 is used to set a corresponding heterogeneous graph convolution layer for each relationship type, and perform a corresponding heterogeneous graph convolution operation on each relationship type in the heterogeneous graph convolution training, generate a corresponding edge prediction result to obtain a triple prediction output for each relationship type, and the triple specifically includes a head entity representation, a relationship type, and a tail entity representation.

[0076] In an embodiment of the present invention, the heterogeneous graph construction module 205 is also used to adopt a multi-head attention mechanism to fuse the entity position information of each entity and the third feature word vector, and use the obtained fused feature vector as the initial feature vector of the node corresponding to the current entity; construct a dictionary form of each entity node in the heterogeneous graph, the key of the dictionary is the type of entity, and the value of the dictionary is a set of initial feature vectors of all entities under the entity type.

[0077] In an embodiment of the present invention, the relationship classification module 206 is used to use the heterogeneous graph and the initial feature vectors of all entity nodes in the heterogeneous graph as input data for each relationship type, and use the heterogeneous graph convolution operation corresponding to the current relationship type to process the input data to obtain a triple prediction result of the current relationship type.

[0078] Furthermore, the relationship classification module 206 is also used in heterogeneous graph convolution training to aggregate the initial feature vectors of the adjacent nodes of each entity node through the process of message passing on the graph to obtain the context information of the current node and perform mean aggregation to obtain a new feature representation, and use the new feature representation after mean aggregation to fuse with the initial feature vector of the current node, and update the feature representation of the current node according to the fused features.

[0079] In addition, an embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for constructing the above-mentioned entity relationship joint extraction model based on heterogeneous graph neural network are implemented.

[0080] In this embodiment, the method for constructing the entity relationship joint extraction model based on the heterogeneous graph neural network can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0081] In addition, an embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method for constructing the entity relationship joint extraction model based on the heterogeneous graph neural network are implemented.

[0082] Figure 4 The overall architecture diagram of the entity relationship joint extraction model based on heterogeneous graph neural network proposed in the embodiment of the present invention, the entity relationship joint extraction model based on heterogeneous graph neural network and the construction method provided in the embodiment of the present invention, introduces the relationship information as a priori knowledge into the dynamic feature word vector of the original text sequence that fuses the contextual semantic information at the word granularity by adopting an adaptive multi-head attention mechanism, and obtains the feature word vector that fuses the contextual semantic information and the relationship information. Moreover, when constructing a heterogeneous graph based on the entity type, virtual relationship edges are added between the entity pairs existing in the heterogeneous graph, and the relationship between the two connected entities is represented by the virtual relationship edges, thereby realizing the joint modeling of entities and relationships. The joint modeling can better capture the contextual dependency between entities and relationships and the relationship interaction between multiple entities. The joint modeling of these relationships can also better restore the complexity of the actual relationships, and more efficiently improve the consistency and accuracy of the overlapping relationships between the extracted entities. Furthermore, the present invention adopts a multi-headed annotated named entity recognition method, which can solve the difficult problems such as entity nesting and overlapping relationships in the Chinese entity relationship joint extraction task at one time, and can more accurately and efficiently identify entity relationship triplets in unstructured text sequences, ensuring that all important information in complex texts can be more fully extracted.

[0083] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0084] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, any one of the claimed embodiments may be used in any combination.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing an entity relationship joint extraction model based on heterogeneous graph neural network, characterized in that: The method comprises: The original text sequence is input into the ERNIE3.0 pre-training model for pre-training to obtain the first dynamic feature word vector of the original text sequence that integrates contextual semantic information at the word granularity; A conditional attention mechanism is used to introduce the predefined relationship type in the original text sequence as prior knowledge into the first dynamic feature word vector, thereby obtaining a second feature word vector that integrates contextual semantic information and relationship type information; Input the second feature word vector into a preset BILSTM model for encoding operation to obtain a forward hidden state sequence and a reverse hidden state sequence, merge the forward hidden state sequence and the reverse hidden state sequence to obtain a bidirectional encoding representation of the second feature word vector, and generate a third feature word vector according to the bidirectional encoding representation of the second feature word vector; The third feature word vector is converted into a preset table output, and a multi-head selection method is used to perform entity annotation on the entity position and entity type in the original text sequence in the table output; According to the table output after entity annotation, a heterogeneous graph is constructed with different entity types as different types of nodes, wherein the initial feature vector of each node includes the entity position information and the fused feature vector of the third feature word vector, and virtual relationship edges are added between entity pairs existing in the heterogeneous graph; A corresponding heterogeneous graph convolution layer is set for each relationship type, and the corresponding heterogeneous graph convolution operation is performed on each relationship type in the heterogeneous graph convolution training to generate the corresponding edge prediction results to obtain the triple prediction output of each relationship type. The triplet specifically includes the head entity representation, the relationship type and the tail entity representation.

2. The method according to claim 1, characterized in that Using the conditional attention mechanism to introduce the predefined relationship types in the original text sequence as prior knowledge into the first dynamic feature word vector includes: Project the first dynamic feature word vector to the value, key, and query vector spaces respectively through a linear layer; Mapping each predefined relationship type into a relationship embedding vector with the same dimension as the query vector through an embedding layer, and adding the relationship embedding vector to the query vector to obtain a query vector that integrates the relationship type information; Split the query vectors of value, key, and fused relation type information into multiple heads respectively; Calculate the dot product of the query vector and the key vector that incorporate the relationship type information to obtain the attention score matrix energy; Apply the softmax function to normalize the attention score matrix to obtain the attention weight attention, use the attention weight to weight the sum of the value vector, and use the obtained attention output as the second feature word vector that integrates contextual semantic information and relationship information.

3. The method according to claim 1, characterized in that The third feature word vector is formed according to the bidirectional encoding representation of the second feature word vector, including: The bidirectional encoding representation of the second feature word vector is feature fused through a fully connected layer to form a third feature word vector.

4. The method according to claim 1, characterized in that: The third feature word vector is converted into a preset table output, including: The output of the tensor shape (batch batch_size, sequence length seq_len, hidden dimension hidden_dim) of the third feature word vector is expanded to seq_len size by adding a new dimension at the third position, and generates an intermediate tensor output in the form of (batch_size, seq_len, seq_len, hidden_dim) to obtain the first hidden state combination matrix of the entity start position at each word granularity; The obtained intermediate tensor output is transposed and the positions of the first dimension and the second dimension are exchanged to obtain the second hidden state combination matrix of the entity end position at each word granularity; The first hidden state combination matrix and the second hidden state combination matrix are concatenated on the last dimension representing the feature to obtain a tabular output.

5. The method according to claim 4, characterized in that The multi-head selection method is used to label the entity positions and entity types in the original text sequence in the tabular output, including: The concatenated hidden state combination output in table form is mapped to the preset label space through the fully connected layer, and the softmax activation function is used to map the concatenated hidden state combination to the final label probability to obtain the entity position and entity type information in the original text sequence.

6. The method according to claim 1, characterized in that The method further comprises: A multi-head attention mechanism is used to fuse the entity position information of each entity and the third feature word vector, and the obtained fused feature vector is used as the initial feature vector of the node corresponding to the current entity; Construct a dictionary form for each entity node in the heterogeneous graph, where the key of the dictionary is the type of entity and the value of the dictionary is a set of initial feature vectors of all entities under the entity type.

7. The method according to claim 1, characterized in that The corresponding heterogeneous graph convolution operation is performed on each relationship type in the heterogeneous graph convolution training to generate corresponding edge prediction results to obtain triple prediction outputs of each relationship type, including: For each relationship type, the heterogeneous graph and the initial feature vectors of all entity nodes in the heterogeneous graph are used as input data, and the heterogeneous graph convolution operation corresponding to the current relationship type is used to process the input data to obtain the triple prediction result of the current relationship type.

8. The method according to claim 7, characterized in that The method further comprises: In heterogeneous graph convolution training, through the process of message passing on the graph, the initial feature vectors of each entity node's adjacent nodes are aggregated to obtain the context information of the current node and perform mean aggregation to obtain a new feature representation. The new feature representation after mean aggregation is fused with the initial feature vector of the current node, and the feature representation of the current node is updated according to the fused features.

9. A joint entity relationship extraction model based on heterogeneous graph neural network, characterized in that: The model includes a named entity recognition unit and a relationship classification unit. The named entity recognition unit includes a pre-training module, a relationship information fusion module, a global feature recognition module and an entity labeling module. The relationship classification unit includes a heterogeneous graph construction module and a relationship classification module, wherein: A pre-training module is used to input the original text sequence into the ERNIE3.0 pre-training model for pre-training, and obtain the first dynamic feature word vector of the original text sequence that integrates contextual semantic information at the word granularity; A relation information fusion module, used to introduce the predefined relation type in the original text sequence as prior knowledge into the first dynamic feature word vector by using a conditional attention mechanism, so as to obtain a second feature word vector that integrates contextual semantic information and relation type information; A global feature recognition module, used for inputting the second feature word vector into a preset BILSTM model for encoding operation to obtain a forward hidden state sequence and a reverse hidden state sequence, merging the forward hidden state sequence and the reverse hidden state sequence to obtain a bidirectional encoding representation of the second feature word vector, and generating a third feature word vector according to the bidirectional encoding representation of the second feature word vector; An entity labeling module, used for converting the third feature word vector into a preset table output, and using a multi-head selection method to label entity positions and entity types in the original text sequence in the table output; A heterogeneous graph construction module is used to construct a heterogeneous graph with different entity types as different types of nodes according to the table output after entity annotation, wherein the initial feature vector of each node includes the entity position information and the fused feature vector of the third feature word vector, and virtual relationship edges are added between entity pairs existing in the heterogeneous graph; The relationship classification module is used to set the corresponding heterogeneous graph convolution layer for each relationship type, and perform the corresponding heterogeneous graph convolution operation on each relationship type in the heterogeneous graph convolution training, generate the corresponding edge prediction results to obtain the triple prediction output of each relationship type, and the triple specifically includes the head entity representation, the relationship type and the tail entity representation.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images