Knowledge graph completion method based on deep semantics and entity category information extraction

Abstract

The invention relates to a knowledge graph completion method based on deep semantics and entity category information extraction, and belongs to the field of knowledge graph completion. According to the method, a DSET model of an end-to-end neural network is provided, and an entity of a predicted knowledge graph missing position is output based on the DSET model; the DSET model comprises a type similarity graph attention encoder TS-GAT which focuses on modeling of entity type information, and a deep semantic information three-dimensional convolution decoder DS-Conv3 which is responsible for extracting deep semantic features of entities and relationships. According to the method, the problems that entity category information is neglected and deep semantic feature extraction is insufficient in a traditional knowledge graph completion method are solved.

Description

Technical Field

[0001] The present invention belongs to the field of knowledge graph completion, and specifically relates to a knowledge graph completion method based on deep semantics and entity category information extraction. Background Art

[0002] Knowledge Graph (KG) is a structured knowledge representation that aims to organize and manage information in a graph structure. It is directed and heterogeneous. The core components of the knowledge graph are entities and relationships, which constitute the basic structural units in the knowledge graph in the form of triples in the form of (subject, relationship, object), such as (Einstein, belongs to, scientist). This representation method intuitively expresses the semantic relationship between entities, that is, Einstein is a scientist. The knowledge graph stores a large amount of world knowledge through such structured data and supports complex queries and reasoning.

[0003] Knowledge Graph Completion (KGC) refers to using existing knowledge graph data to predict and fill in missing entities or relationships. This process is crucial to improving the quality and integrity of the knowledge graph, so that the knowledge graph can better support various applications. Existing knowledge graph completion models are mainly divided into four categories: translation models, tensor decomposition models, convolutional neural network models, and graph neural network models:

[0004] 1. Translation model: The TransE[1] model assumes that each relation in the knowledge graph can be viewed as a linear transformation between the head entity vector and the tail entity vector. TransE encountered challenges in handling complex relations, while TransH[2] introduced the concept of relational hyperplane to improve this problem. TransR[3] further overcomes the limitation that entities and relations are in different semantic spaces, and each relation has an independent space for translation. TransD[4] develops the idea of ​​TransR, dynamically generates independent mapping matrices for each entity and relation, and enhances the flexibility of the model in handling complex relational data. RotatE[5] simulates symmetry, antisymmetry, and transitivity through rotation in complex space; 2. Tensor decomposition model: The RESCAL[6] model represents entities as vectors and relations as matrices, and predicts by multiplication. DistMult[7] simplifies the relational representation of RESCAL, improves computational efficiency, and retains representation capabilities. ComplEx[8] introduces embedding of complex domains to enhance the expression of symmetric and antisymmetric relations. SimplE[9] uses CP decomposition to introduce head and tail embeddings and two types of relationship embeddings, which improves prediction accuracy. Third, convolutional neural network model: ConvE

[10] uses a two-dimensional convolutional neural network to learn the embedding representation of entities and relationships, which improves the accuracy of link prediction. On the other hand, ConvKB

[11] directly applies a one-dimensional convolution operation on triple embeddings to learn complex interactive relationships. InteractE

[12] improves ConvE by introducing more complex feature interactions and embedding mechanisms. Fourth, graph neural network model: R-GCN

[13] introduces independent specific weights for each type of relationship, thereby effectively handling relationship diversity and improving model accuracy, and introduces a self-connection mechanism to enhance the understanding of entity self-attributes. SACN

[14] combines structured attention with graph convolutional networks to selectively aggregate neighboring node information, significantly improving the accuracy of knowledge graph completion. KBGAT

[15] dynamically captures relationship importance through a graph attention network and assigns unique attention weights to each connection. CompGCN

[16] combines entity and relation embeddings into the graph convolution process and updates the relation embeddings at the same time, thus enhancing the expressiveness of the embeddings.

[0005] Knowledge graphs provide good support for many intelligent applications such as intelligent question answering

[17] , intelligent recommendation

[18] , and semantic parsing

[19] . Currently, many large-scale knowledge graphs have been developed, such as YAGO

[20] , WIKI

[21] , WordNet

[22] , Freebase

[23] , and ICEWS

[24] , but they still have the problem of incompleteness, which means that a large number of potential entities and relationships have not yet been discovered. Take Freebase as an example. There is a lot of missing information in it

[25] . For example, in terms of "nationality", "education" and "parents", 75%, 91% and 94% of the person entities lack this information respectively. This incompleteness of the knowledge graph seriously limits the efficiency of downstream applications. Therefore, knowledge graph completion (KGC) has become a research hotspot in the current knowledge graph field. KGC aims to use existing data or knowledge to predict and complete the missing entities or relationships in the knowledge graph through inference or learning

[26] , thereby improving the completeness and utilization value of the knowledge graph.

[0006] Historically, research on knowledge graph completion has mainly focused on structured representation learning, attempting to mine and predict missing information in knowledge graphs by learning low-dimensional embeddings of entities and relationships. Among them, the neural network models that have emerged in recent years can learn the deep features and mapping relationships of data through training, so as to better understand and process complex entity relationships. The graph neural network model can make full use of the structural information of the knowledge graph, effectively integrate nodes and their contextual information, and thus capture the complex relationships between nodes.

[0007] In addition, in order to simultaneously complete the tail entity prediction and head entity prediction tasks of knowledge graph completion, most existing methods such as ConvE, InteractE, and CompGCN will create the inverse relationship of the relationship for the knowledge graph, that is, from the existing triples (e s ,r,e o ) generates the corresponding inverse relation triple (e o ,r',e s ), and add the generated new triples to the training set, validation set, and test set to correctly evaluate the model performance. This workflow is based on a simple assumption: if a relationship holds in one direction, then its inverse relationship also holds in the opposite direction. This effectively increases the number of training samples and enhances the learning and expression capabilities of the model to a certain extent. However, the existing models have the following limitations:

[0008] First, the category similarity between entities is ignored, that is, the entity category information is ignored. Entity category information plays an important role in the knowledge graph and is crucial for reasoning about missing entities. Taking the "born in" relationship in the real world as an example, the reasonable connection should be "person" and "place". If the entity category information is ignored, the model may make wrong predictions, such as mistakenly connecting the "person" and "born in" relationship with another "person". From the perspective of the categories of the entities connected by the relationship, the relationships in the knowledge graph can be divided into two types: one is an equivalence relationship, which means that the categories of the head and tail entities of the relationship are consistent, and typical examples are "classmates" and "colleagues"; the second is a non-equivalence relationship. In this case, the categories of the head and tail entities of the relationship are not the same, such as "born in", "nationality" and other relationships. In fact, in the equivalence relationship, there is potential category information, and the head and tail entities can be regarded as entities of the same category. Aggregating the head and tail entity information connected by these equivalence relationships can effectively model and reveal the entity category information in the knowledge graph. Although current graph neural network models, such as R-GCN and CompGCN, also have a central entity aggregation process, they ignore the entity category information in the equivalence relation and do not include it in the scope of message passing. Therefore, they cannot explicitly model the entity category information.

[0009] 2. Insufficient extraction of deep semantic features. On the one hand, in the current traditional knowledge graph completion model, entities and relationships are represented by low-dimensional vectors. However, such methods are difficult to capture and express the deep and complex semantics of entities and relationships. On the other hand, while the current neural network model improves the interaction between entity and relationship embeddings, it cannot retain their respective local features, resulting in the loss of deep semantics. For example, ConvE converts the knowledge graph completion task into a two-dimensional input that can be processed by a convolutional neural network to extract the local features of entities and relationships, but the interaction between entities and relationships is insufficient. InteractE adopts a complex interaction method of random feature arrangement, and further increases the interaction between entity embedding and relationship embedding on the basis of ConvE, thereby improving the performance of knowledge graph completion, but the extraction of local features of entities and relationships is lost. The current KGC model has not yet proposed a solution for extracting interaction features and local features at the same time.

[0010] Therefore, how to achieve efficient completion of static knowledge graphs by deeply mining entity category information and deep semantic information in the knowledge graph is the key research content of the present invention. Summary of the invention

[0011] The purpose of the present invention is to solve the problem of neglecting entity category information and insufficient extraction of deep semantic features in traditional knowledge graph completion methods, and to provide a knowledge graph completion method based on deep semantics and entity category information extraction.

[0012] To achieve the above objectives, the technical solution of the present invention is: a knowledge graph completion method based on deep semantics and entity category information extraction, proposing an end-to-end neural network DSET model, and outputting predicted entities at missing positions in the knowledge graph based on the DSET model.

[0013] In one embodiment of the present invention, the DSET model includes: a type similarity graph attention encoder TS-GAT that focuses on modeling entity type information, and a deep semantic information three-dimensional convolutional decoder DS-Conv3 that is responsible for extracting deep semantic features of entities and relationships.

[0014] In one embodiment of the present invention, the type similarity graph attention encoder TS-GAT considers the intrinsic connection between entity types in equivalence relations, and identifies equivalence relations by calculating the similarity of positive and negative relationship embeddings during the aggregation process, thereby giving similar entities higher association weights; at the same time, TS-GAT also considers the diversity information between entities, and aggregates triple-level information with both entities and relationships for the central entity.

[0015] In one embodiment of the present invention, the deep semantic information three-dimensional convolution decoder DS-Conv3 uses Gaussian function mapping to capture the deep and complex semantics of entities and relationships, and uses three-dimensional convolution operations to extract local features of entities and relationships, while capturing the complex interaction patterns between them.

[0016] In one embodiment of the present invention, the type similarity graph attention encoder TS-GAT proposes a head-tail type attention mechanism, which determines whether it is an equivalent relationship by calculating the cosine similarity between the forward and reverse relationship embeddings corresponding to the relationship, and determines whether the head and tail entities belong to the same category; the specific implementation is as follows:

[0017] First, the calculation formula for the positive and negative relationship similarity is:

[0018]

[0019] The symbol · represents the dot product of two vectors, and ||r||2 and ||r'||2 represent the bi-norm of vectors r and r', respectively. Since the value range of cosine similarity is between -1 and 1, in order to facilitate the subsequent attention score calculation, the value range of positive and negative relationship similarity is controlled between 0 and 1;

[0020] Next, calculate the attention score between the head and tail entities:

[0021]

[0022] Among them, α ij Representing entity e i and entity e jThe type attention score between ij Represents entity e i and entity e j The embedding vector of the relationship between ij 'Represents entity e i and entity e j The embedding vector corresponding to the reverse relationship between It is entity e i The neighbor relationship set of

[0023] Next, the embedding vectors of the neighbor entities are weighted according to the attention scores and updated; the passed messages are calculated and updated in the following way:

[0024]

[0025] in, Indicates that the neighbor entity is passed to entity e i Type information, Represents all and entity e i A collection of connected entities, It is the mapping matrix of type information.

[0026] In one embodiment of the present invention, the type similarity graph attention encoder TS-GAT also adopts a diverse information transmission process based on the attention mechanism, so that the central entity can obtain discriminative information from different connected triples, which is specifically implemented as follows:

[0027] The entity similarity attention score is used to measure the similarity between the neighbor entity and the central entity. The calculation formula is as follows:

[0028]

[0029] Among them, β ij It is entity e i About Entity e j Through the relationship ij Entity similarity attention score of ;

[0030] After calculating the entity similarity attention scores of each neighbor entity, TS-GAT will perform weighted aggregation of the information of each neighbor entity based on these scores. In the aggregation process, not only the neighbor entity embeddings but also the relationship embeddings between them are considered. The two embeddings are fused by Hadamard product to obtain an updated representation of the diversity information. The calculation formula is as follows:

[0031]

[0032] in, Represents the entity e iDiversity information, Represents all entities e i The connected fact set, ⊙ represents the Hadamard product, It is the mapping matrix of diversity information.

[0033] In one embodiment of the present invention, the update mechanism of the type similarity graph attention encoder TS-GAT is defined as follows:

[0034]

[0035] in, is a learnable parameter matrix, σ is the nonlinear activation function ReLU, e' i is the updated entity embedding, which will be used as the input entity embedding for the next iteration until the entire model training process is completed.

[0036] In one embodiment of the present invention, the deep semantic information three-dimensional convolution decoder DS-Conv3 designs a multi-dimensional Gaussian mapping method to perform mapping conversion on the embedding of entities and relationships, as follows:

[0037] The mapping function for the entity is composed of a series of Gaussian functions φ i (e) where each kernel function focuses on the input embedding and a specific center point c i The similarity between them; specifically, for the i-th kernel function, we get:

[0038]

[0039] where c i represents the i-th Gaussian center, which can be obtained by model parameterization learning, σ i is the corresponding standard deviation; the mapping function covers d Gaussian functions, forming a mapping function Φ(e):

[0040] Φ(e)=[φ1(e),φ2(e),…,φ d (e)]

[0041] Through the mapping function Φ, the head entity embedding vector e s is converted into a new feature vector Similarly, the relation embedding r is also transformed into a new feature vector r Φ :

[0042]

[0043] In one embodiment of the present invention, the deep semantic information three-dimensional convolution decoder DS-Conv3 also depicts the high-dimensional complex semantics of entities and relationships and their interactive information through three-dimensional convolution operations, which is specifically implemented as follows:

[0044] Given the original head entity and relation embedding e s , r, and the embedding vector obtained by Gaussian mapping r Φ , the feature input is constructed as follows:

[0045]

[0046] Among them, ρ represents the reshaping operation, which reshapes the embedded vector into a two-dimensional feature map with a height of h and a width of w. Stack represents the stacking operation, and axis is the dimension in which the stacking operation is performed. F has a dimension of h×w×5, which represents the feature input tensor.

[0047] Then use a three-dimensional convolution kernel to calculate the feature input tensor:

[0048] X = σ(Conv3D(F))

[0049] Among them, X represents the output feature vector after the three-dimensional convolution extracts the interactive features and local features, σ represents the nonlinear activation function ReLU, Conv3D represents the three-dimensional convolution operation, and the size of the convolution kernel is set to c×k h ×k w ×2, c represents the number of channels, k h , k w Represent the height and width of the convolution kernel respectively, and its depth dimension is 2;

[0050] Finally, the feature map obtained after the convolutional layer is converted into a one-dimensional feature vector using the flattening operation vec():

[0051] Out = vec(X).

[0052] The present invention also provides a knowledge graph completion system based on deep semantics and entity category information extraction, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. In response to the problem that existing models in this field ignore entity category information, this paper designs a novel type similarity graph attention encoder to deeply analyze the similarity and equivalence relationship between entities, and models entity type information in the graph context entity aggregation process. This process helps the model understand and distinguish different types of entities and their relationships.

[0055] 2. In view of the fact that the existing models in this field are not good at extracting deep semantics, this paper designs a new three-dimensional convolution decoder for deep semantic information, which uses Gaussian function mapping technology to capture the deep semantic information of entities and relationships. This method not only enhances the model's understanding of the complex interaction features between entities. In addition, through the stacking operation and the application of three-dimensional convolution kernels, the accurate extraction of interaction features between various embeddings is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flow chart of a knowledge graph completion method based on deep semantics and entity category information extraction of the present invention. DETAILED DESCRIPTION

[0057] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0058] The present invention proposes a knowledge graph completion method based on deep semantics and entity category information extraction, proposes an end-to-end neural network DSET model, and outputs the predicted entities at the missing positions of the knowledge graph based on the DSET model. The DSET model includes: a type similarity graph attention encoder TS-GAT that focuses on modeling entity type information, and a deep semantic information three-dimensional convolution decoder DS-Conv3 that is responsible for extracting deep semantic features of entities and relationships.

[0059] The present invention also provides a knowledge graph completion system based on deep semantics and entity category information extraction, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps described above can be implemented.

[0060] The following are specific implementation examples of the present invention.

[0061] 1. Overall idea and pre-definition

[0062] The DSET model aims to solve the problem of neglecting entity category information and insufficient extraction of deep semantic features in traditional knowledge graph completion methods. Figure 1As shown in the figure, the DSET model is an end-to-end neural network that contains two core components: (1) Type Similarity Graph Attention Encoder TS-GAT: This encoder focuses on modeling entity type information. This module considers the intrinsic connection between entity types in equivalence relations. In the aggregation process, it identifies equivalence relations by calculating the similarity of positive and negative relation embeddings, thereby giving similar entities a higher association weight. At the same time, TS-GAT also considers the diversity information between entities, so it also aggregates triple-level information with both entities and relations for the central entity; (2) Deep Semantic Information Three-Dimensional Convolution Decoder DS-Conv3D: The decoder is responsible for extracting deep semantic features of entities and relations. First, a Gaussian function mapping is used to capture the deep and complex semantics of entities and relations, and a three-dimensional convolution operation is used to extract local features of entities and relations, while capturing the complex interaction patterns between them. Combining TS-GAT and DS-Conv3D, the DSET model outputs predicted missing entities. By calculating the matching scores between candidate entities and existing entities and relations, the model predicts the entity that is most likely to fill the missing position in the knowledge graph.

[0063] The relevant definitions of the present invention are given below.

[0064] Definition 1 (Knowledge Graph, ) in They represent entity sets, relationship sets, and fact triple sets respectively.

[0065] Definition 2 (Entity Set, ) represents the set of all entities in the knowledge graph, e s ,e o Represent the subject entity and the object entity respectively.

[0066] Definition 3 (Relation set, ) Represents the set of all relations r in the knowledge graph.

[0067] Definition 4 (Fact Set, ) Represents a set of triple facts.

[0068] Definition 5 (Embedding Dimension) d represents the dimension of the embedding vector in the model

[0069] Definition 6 (Entity Embedding) denote the embeddings corresponding to the subject and object entities, respectively.

[0070] Definition 7 (Relation Embedding) Represents the embedding of relation r.

[0071] 2. DSET model design

[0072] 2.1 Type Similarity Graph Attention Encoder (TS-GAT)

[0073] When performing knowledge graph completion tasks, entity type information is crucial for reasoning about possible head and tail entities under a specific relationship. Currently popular graph neural network models, such as R-GCN and CompGCN, all use a central entity aggregation mechanism when processing knowledge graphs. These models update the representation of central nodes by aggregating information from neighboring nodes to capture and integrate structural information in the graph. However, a significant limitation of these models is that they ignore the entity category information contained in equivalence relations and do not fully consider the importance of category information in the process of interaction and information transfer between entities. This defect limits the performance of the model in inferring potential entities.

[0074] To address this challenge, the present invention designs a novel Type-Similarity Graph Attention Encoder (TS-GAT). TS-GAT is dedicated to enhancing the entity aggregation process. Its uniqueness lies in the introduction of a head-tail type attention score for the first time to identify equivalent relations. TS-GAT can analyze and dynamically adjust the connection weights between entities based on the head-tail type attention scores, thereby more accurately capturing similar characteristics between entities of the same type. Such a process significantly improves the model's ability to recognize and model entity categories in knowledge graphs. At the same time, TS-GAT also takes into account the differences between entities, and achieves a more comprehensive understanding of entities by aggregating diverse information including neighbor entities and relationships for the central entity.

[0075] 2.1.1 Type Information Transfer

[0076] The explicit modeling of entity types in TS-GAT comes from the observation of equivalence relations, that is, the head and tail entity types connected by equivalence relations are consistent. At the same time, the positive and negative relations of equivalence relations should have high semantic similarity. Therefore, capturing equivalence relations can share similar information between the head and tail entities of equivalence relations, thereby modeling entity types.

[0077] The present invention proposes a head-tail type attention mechanism. By calculating the cosine similarity between the forward and reverse relationship embeddings corresponding to the relationship, it is possible to determine whether it is an equivalent relationship and whether the head and tail entities belong to the same category. First, the calculation formula for the forward and reverse relationship similarity is:

[0078]

[0079] Here, the symbol · represents the dot product of two vectors, and ||r||2 and ||r'||2 represent the bi-norm of vectors r and r', respectively. Since the value range of cosine similarity is between -1 and 1, in order to facilitate the subsequent attention score calculation, the value range of the positive and negative relationship similarity is controlled between 0 and 1. Next, the attention score between the head and tail entities is further calculated:

[0080]

[0081] Among them, α ij Representing entity e i and entity e j The type attention score between ij Represents entity e i and entity e j The embedding vector of the relationship between ij ' represents the embedding vector of its corresponding reverse relation, It is entity e i The neighbor relationship set of .

[0082] From the above formula, we can see that when the similarity between the positive relationship and the negative relationship of a relationship is greater, the possibility that the relationship is an equivalent relationship is greater, and the attention score is also greater. According to this attention score, the model can dynamically adjust the connection weights between entities, so that the weight of similar entities when transmitting type information becomes larger. Next, the embedding vector of the neighbor entity is weighted according to the attention score and updated. The transmitted message can be calculated and updated in the following ways:

[0083]

[0084] here, Indicates that the neighbor entity is passed to entity e i Type information, Represents all and entity e i A collection of connected entities, It is the mapping matrix of type information.

[0085] 2.1.2 Diversity Information Transmission

[0086] In addition to emphasizing the importance of entity type information, TS-GAT also pays attention to the diversity between entities. In order to more comprehensively represent the different characteristics of entities and their interactions with relationships, TS-GAT introduces a diversity information transmission mechanism to more carefully capture the similarities and differences between entities and characterize their unique roles in the knowledge graph.

[0087] Therefore, the present invention adopts a diversity information transmission process based on the attention mechanism, so that the central entity can obtain discriminative information from different connected triples. Like type information transmission, diversity information transmission also uses the attention mechanism to determine the weights of neighbor entities, but emphasizes the differences between entities when calculating the weights. Specifically, the model uses the entity similarity attention score to measure the similarity between the neighbor entity and the central entity, and the calculation formula is as follows:

[0088]

[0089] Among them, β ij It is entity e i About Entity e j Through the relationship ij Entity similarity attention score.

[0090] After calculating the entity similarity attention scores of each neighbor entity, TS-GAT will perform weighted aggregation of the information of each neighbor entity based on these scores. The aggregation process not only considers the neighbor entity embedding, but also the relationship embedding between them. The two embeddings are fused by Hadamard product to obtain an updated representation of the diversity information. This approach can bring complete triple-level information to the central entity and enrich the diversity of entity representation. The calculation formula is as follows:

[0091]

[0092] Here, Represents the entity e i Diversity information, Represents all entities e i The connected fact set, ⊙ represents the Hadamard product, It is the mapping matrix of diversity information.

[0093] 2.1.3 Entity Aggregation and Update

[0094] The entity aggregation and update process of TS-GAT combines the type information transmission described in Section 2.1.1 with the diversity information transmission mentioned in Section 2.1.2. On this basis, in order to incorporate type messages and diversity messages, the original entity embedding also needs to be integrated with the information of neighbor entities through a special update strategy. First, each entity collects the type information transmitted from its neighbors. and diversity information In addition, the entity retains its original embedding i , which represents the initial state of the entity without the influence of external information. Subsequently, TS-GAT will combine this information to update the embedding of the entity. The specific update mechanism is defined as follows:

[0095]

[0096] in, is a learnable parameter matrix, and σ is the nonlinear activation function ReLU. e' i is the updated entity embedding, which will be used as the input entity embedding for the next iteration until the entire model training process is completed.

[0097] Through this process, TS-GAT can not only capture the type attributes of entities, but also depict the complex relationships between entities and their neighbors, and extract key diversity information that affects knowledge graph reasoning. Combining the dual attention mechanism of type and diversity information, TS-GAT can more comprehensively understand and represent the entities and relationships of knowledge graphs, thereby improving the effectiveness of knowledge graph completion tasks.

[0098] 2.2 Deep Semantic Extraction 3D Convolutional Decoder (DS-Conv3D)

[0099] In order to extract the deep semantic information of entities and relationships from the neural network and make full use of this information to complete the knowledge graph completion task, the present invention designs a novel deep semantic extraction 3D convolutional decoder (DeepSemantic Extraction3D Convolutional Decoder, DS-Conv3D). DS-Conv3D uses Gaussian function mapping and improves the low-dimensional embedding of entities and relationships through nonlinear transformation to capture deep semantic information. In addition, compared with the existing two-dimensional convolution decoder, the DS-Conv3D of the present invention shows more efficient and sophisticated performance. With the help of stacking operations in three-dimensional space and three-dimensional convolution kernels designed to capture complex data patterns, DS-Conv3D can capture the complex interaction patterns between entities and relationships, and can maintain a high degree of sensitivity and maintenance to the unique local features of entities and relationships.

[0100] 2.2.1 Gaussian function mapping

[0101] The present invention designs a multi-dimensional Gaussian mapping method to map and transform the embedding of entities and relations. After such transformation, the original embedding space is decoupled into an independent feature space representing deep semantic attributes. Taking the mapping function of the entity as an example, the mapping is composed of a series of Gaussian functions φ i (e) where each kernel function focuses on the input embedding and a specific center point c i Specifically, for the i-th kernel function, we can get:

[0102]

[0103] where ci represents the i-th Gaussian center, which can be obtained by model parameterization learning, σ i is the corresponding standard deviation, which is also part of the model parameters. The mapping function covers d such Gaussian functions, thus forming a mapping function Φ(e):

[0104] Φ(e)=[φ1(e),φ2(e),…,φ d (e)]

[0105] Through this mapping function Φ, the head entity embedding vector e s is converted into a new feature vector Similarly, the relation embedding r is also transformed into a new feature vector r Φ :

[0106]

[0107] r Φ =Φ(r)

[0108] This Gaussian mapping method can reveal nonlinear structures and deep patterns in embeddings and facilitate capturing rich semantic information of entities and relations. By adopting this technique, the model can more effectively infer and generalize complex information in knowledge graphs.

[0109] 2.2.2 Three-dimensional convolution operation

[0110] In the design of the DS-Conv3D decoder, the three-dimensional convolution operation is one of the core links, which is used to characterize the high-dimensional complex semantics of entities and relationships and their interactive information. The following is a detailed introduction to the implementation steps of the three-dimensional convolution operation. First, it is necessary to construct the input features, that is, to reshape the embedding vector mapped by the Gaussian function and the original embedding vector and use the stacking operation in the depth dimension. Given the original head entity and relationship embedding e s , r, and the embedding vector obtained by Gaussian mapping r Φ , the feature input is constructed as follows:

[0111]

[0112] Where ρ represents the reshape operation, which reshapes the embedding vector into a two-dimensional feature map with a height of h and a width of w. Stack represents the stacking operation, and axis is the dimension in which the stacking operation is performed. F has a dimension of h×w×5, which represents the feature input tensor. Then a three-dimensional convolution kernel is used to calculate the feature input tensor:

[0113] X = σ(Conv3D(F))

[0114] Where X represents the output feature vector after the three-dimensional convolution extracts the interactive features and local features. σ represents the nonlinear activation function ReLU. Conv3D represents the three-dimensional convolution operation, and the size of the convolution kernel is set to c×k h ×k w ×2, c represents the number of channels, k h , k w They represent the height and width of the convolution kernel respectively, and its depth is 2.

[0115] The 3D convolution operation can capture the interactions between entity and relation embeddings at different embedding levels, including the original features and the deep features mapped by Gaussian functions. Using a convolution kernel with a depth of 2 is particularly beneficial for extracting such interactive features because it can process two adjacent feature map layers at the same time. For a given feature map stack F, the two layers spanned by the 3D convolution kernel in the depth direction can be equivalent to processing the following four feature pair interactions: the original head entity embedding e s With the original relation embedding r, the original head entity embedding e s and the relational embedding r after Gaussian mapping Φ , Head entity embedding after Gaussian mapping and the relational embedding r after Gaussian mapping Φ , Head entity embedding after Gaussian mapping Embed r with the original relation;

[0116] The three-dimensional convolution kernel is k in the spatial dimension h ×k w The movement of also allows the model to extract features of each local area. In this way, the model not only integrates the relationship between local features, but also explores the deep interaction between entities and relationships in different dimensions and levels, which is impossible for previous two-dimensional convolution methods.

[0117] Finally, the feature map obtained after the convolution layer needs to be converted into a one-dimensional feature vector using the flattening operation vec() for subsequent tasks:

[0118] Out = vec(X)

[0119] 2.3 Training and Optimization

[0120] 2.3.1 Score Function

[0121] Project the flattened vector Out into the embedding space And match it with the tail entity embedding through the dot product operation, and the score function is defined as follows:

[0122]

[0123] Among them, eo represents the tail entity embedding, W out is a trainable weight matrix.

[0124] 2.3.2 Loss Function

[0125] Model training aims to minimize the gap between the predicted tail entity and the true tail entity. In the process of selecting the tail entity, the predicted probability It is obtained by applying the logistic function sigmoid to the score. To train the model parameters, the binary cross entropy loss defined below is minimized:

[0126]

[0127] Where N is the number of entities, y i Represents the label of a positive or negative triplet. In the case of a positive triplet, y i The value is 1, and the negative triples are obtained by randomly replacing the subject or object entity. i The value is 0.

[0128] 3. Experiment

[0129] 3.1 Experimental Setup

[0130] 3.1.1 Dataset

[0131] In order to effectively evaluate the knowledge graph completion technology and improve the challenge and fairness of the dataset, the present invention experimentally evaluates the DSET model on two improved benchmark datasets FB15k-237

[27] and WN18RR

[10] . WN18RR is a revised version of WN18 derived from the English vocabulary knowledge base WordNet. Because WN18 has the problem of test set information leakage, some inverse relations that may cause evaluation bias, such as "hypernym" and "hyponym", are removed from WN18RR to increase the difficulty and practicality of the dataset. FB15k-237 is a subset of the Freebase knowledge graph, mainly involving movies, actors, awards, sports and other related content. It contains 14,951 entities and 592,213 relationship triples, and is mainly used for knowledge graph completion research, especially link prediction tasks. The statistical information of each dataset is shown in Table 1:

[0132] Table 1 Dataset details

[0133]

[0134] 3.1.2 Evaluation Metrics

[0135] In order to accurately measure the performance of the model, this paper will discuss two key evaluation indicators: Mean Reciprocal Rank (MRR) and hit rate @N (Hits@N). These two indicators are widely used in knowledge graph completion tasks and can provide a detailed and comprehensive evaluation of model performance. In the knowledge graph completion task, given a triple (subject, relation, object) containing a missing subject entity or object entity, the task of the model is to predict the correct entity of the missing part and give a ranking of a list of candidate entities. MRR calculates the average of the reciprocal rankings of the correct answers of all test samples. Specifically, if a model predicts the correct answer for a test sample to be ranked pth, the reciprocal ranking of the sample is 1 / p. The performance of the model for all test samples is expressed as the average of all reciprocal rankings, that is:

[0136]

[0137] Among them, |Q| is the total number of test samples, and rank i is the ranking of the correct answer of the i-th sample. The higher the MRR value, the more accurate the model's prediction ranking is. Therefore, MRR is a powerful indicator for evaluating the overall performance of the model in the knowledge graph completion task.

[0138] Hits@N is another important indicator to measure the predictive ability of the model. For a given test triple, the model provides a ranked list, and if the correct answer appears in the top N positions, it is considered a "hit". Therefore, Hits@N reflects the probability that the correct answer is in the top N positions. Usually, researchers will choose different N values, such as 1, 3 or 10, to observe the performance of the model under different strictness indicators. A higher Hits@N value means that the model is more accurate and reliable in predicting the top N. The hit rate calculation formula can be expressed as:

[0139]

[0140] 3.1.3 Baseline

[0141] In this paper, the DSET model is compared with the most advanced models in the field of static knowledge graph completion. The comparison covers translation-based models, TransE, RotatE, and PairRE

[28] ; tensor decomposition-based models, including DistMult, ComplEx, and TuckER

[29] ; neural network-based models, ConvE, InteractE, JointE

[30] , M-DCN

[31] , MRAN

[32] , and DTAE

[33] ; and graph neural network-based models, SACN and CompGCN.

[0142] 3.1.4 Experimental details

[0143] The DSET model is developed based on the PyTorch framework, and all related experiments are completed on a computing platform equipped with an NVIDIA GeForceRTX 3090 graphics card. The model is initialized by setting the starting parameter values ​​through the Xavier algorithm, and the Adam optimizer is used during training. The learning rate is uniformly set to 0.001. And an early stopping mechanism is implemented to select the model with the best performance on the validation set. When processing the FB15k-237 dataset, the convolution kernel size is set to 7×7×2; on the WN18RR dataset, the convolution kernel size is 5×5×2. On both datasets, the embedding dimension (d) of the model is 400, and 128 convolution filters are set.

[0144] 3.2 Experimental Results

[0145] 3.2.1 Link Prediction

[0146] Link prediction is a key task in knowledge completion, which aims to predict entities and relations that are not directly observed based on known entities and relations. The knowledge graph is a network structure composed of entity nodes, and the edges in the network represent the semantic connections between entities. Link prediction aims to reveal these potential connections, thereby expanding and refining the knowledge graph. The link prediction effect of the DSET model on the FB15K-237 and WN18RR datasets is shown in Table 2. The best and second-best results are marked in bold and underlined in the table, respectively. The results of TransE, DistMult, and ComplEx are derived from the JointE report, while other results are cited from the corresponding original papers.

[0147] The DSET model proposed in this paper surpasses the current leading models on both datasets. On the FB15k-237 dataset, the DSET model improves by 2.2%, 0.4%, 1.8% and 0.9% in MRR, Hits@1, Hits@3 and Hits@10 respectively. On the WN18RR dataset, the improvements of the same series of indicators are 1.5%, 0.4%, 3.4% and 4.9% respectively. CompGCN uses ConvE as a decoder and incorporates an encoder to integrate the contextual information of the graph, thereby achieving significant performance improvement. However, it fails to fully simulate type information and extract deep semantics. In contrast, DSET introduces a new type similarity graph attention mechanism. This mechanism enables the model to effectively identify equivalent relations when aggregating information from the central entity, more accurately characterize entity type information, and thus promote accurate interaction between entities. DSET also uses Gaussian function mapping technology to extract deep semantic features, and effectively captures the complex interactions between entities and relations through three-dimensional convolution operations, maintaining their unique local characteristics. These advantages enable DSET to achieve significant improvements in link prediction tasks, highlighting its advanced capabilities in knowledge graph completion.

[0148] Table 2 Link prediction results for FB15k-237 and WN18RR datasets

[0149]

[0150] 3.2.2 Ablation Experiment

[0151] In order to prove the effectiveness of each module introduced in the model proposed in the present invention for the KG completion task, the present invention designs a series of ablation experiments. In this part, some key components are eliminated one by one and the changes in model performance are observed. Ablation experiments with four settings are mainly carried out: DSET (-G) represents the elimination of the graph encoder, namely the TS-GAT encoder, from the model to verify its importance in capturing the deep relationship between entities and entity type information; DSET (-3D) represents the model using ConvE to replace the deep semantic information three-dimensional convolution decoder DS-Conv3D proposed in the present invention; DSET (-M) represents the removal of Gaussian function mapping and the use of original embedding instead; DSET (-A) represents the removal of type attention, that is, the removal of the type attention scoring mechanism in the TS-GAT encoder, in order to observe the importance of this mechanism in conveying type information to similar entities. The results of the ablation experiment are shown in Table 3.

[0152] First, from the results of the DSET(-G) setting, without the TS-GAT encoder, the effect of the model has significantly decreased compared with the full model. This proves that the TS-GAT encoder plays a key role in capturing the deep relationship between entities and entity type information. Secondly, under the DSET(-3D) setting, it is found that the performance of the model also declines after replacing the decoder with ConvE, indicating that the three-dimensional convolution decoder does a better job in capturing the local features of entities and relationships and deep interaction patterns. Thirdly, in the DSET(-M) setting, it is found that the performance of the model also declines after removing the Gaussian function mapping, which proves that the Gaussian kernel mapping plays a key role in capturing the deep semantic information of entity and relationship embedding. Finally, under the DSET(-A) setting, the model without the type attention mechanism performs poorly, proving that the type attention mechanism plays a very important role in transmitting and updating type information. In summary, from the results of the ablation experiment, it can be seen that each component in the model has an important impact on the final performance of the model, proving the effectiveness and necessity of the model design of the present invention.

[0153] Table 3 Ablation experiment results

[0154]

[0155] 3.2.3 Case Study

[0156] This paper demonstrates the performance and effectiveness of the DSET model in the knowledge graph completion task through a specific example study. Specifically, the present invention will use the FB15k-237 dataset as the research object, and compare the performance difference between the DSET model and the classic graph neural network-based model CompGCN for the link prediction task. Table 4 provides an example of the link prediction task (Jennifer Aniston, friend,?). In this example, the model needs to predict entities that have a "friend" relationship with "JenniferAniston".

[0157] Table 4: Sample study of FB15k-237 dataset, bold indicates the correct candidate answer

[0158]

[0159]

[0160] According to the data provided in Table 4, during the prediction process, the CompGCN model gave three candidate answers, and the highest ranked one was the wrong answer "Friends, season 1". This is because although the CompGCN model can aggregate the contextual information of the graph structure, it cannot model the entity category information, resulting in the wrong predicted entity category; while the DSET model successfully predicted the correct answer "Justin Long". The core reason why the DSET model can successfully predict the correct answer is its explicit modeling ability of equivalence relations and entity category information. The DSET model introduces the head and tail type attention into the graph context entity aggregation process by introducing the TS-GAT encoder. This design enables the model to dynamically adjust the connection weights between entities according to the type information of the head and tail entities during prediction. For the task in this example, DSET uses TS-GAT to identify that "friend" is an equivalence relation with the same head and tail entity type, and that "Jennifer Aniston" and "Justin Long" are both "person" entities and co-starred in the movie "He's Just Not That Into You", thus achieving accurate prediction.

[0161] 3.2.4 Stacking sequence analysis

[0162] The present invention conducts a stacking order analysis experiment, the main purpose of which is to explore the impact of changing the stacking order of feature construction before 3D convolution on model performance. In the depth dimension, the present invention defines the following different stacking orders: (r, e s ,r Φ , r)、(e s ,r, e s ,r)、(r,e s ,e,r Φ , )、(e s ,r,r, r Φ )、(r,e s ,r Φ , r). The present invention conducts experiments with these five different combinations and finds that changes in the stacking order do affect the model performance, which confirms the importance of selecting an appropriate stacking order for capturing deep interactive information between entities and relationships. The experimental results are shown in Table 5.

[0163] Table 5 Stacking sequence analysis experimental results

[0164]

[0165] The experimental results show that among all the experimental combinations, the combination (r, e, r Φ , r) provides the best performance, indicating that including the original embedding and the Gaussian-mapped embedding in the first two layers and using only the original embedding in the last two layers can obtain the best deep interaction information of entities and relations. When the convolution kernel depth is 2, since the convolution kernel can operate across two adjacent feature layers, this combination can capture four interaction features, while other combinations can only capture three or two. Therefore, from the perspective of capturing interaction features, (r,e s ,r Φ , r) This stacking order has obvious advantages, which also explains why this combination performs best in the experiment.

[0166] 3.3 Application Scenarios

[0167] The deep semantic mining feature of the DSET model makes it have wide application potential in many fields, especially in the following scenarios:

[0168] ① Question and answer system:

[0169] Knowledge Question and Answer: The powerful semantic recognition and understanding capabilities of the DEST model can help the question-answering system perform relational reasoning based on its semantic information and category information, thereby answering users' questions more accurately.

[0170] Knowledge graph enhancement: The DSET model can help build and enhance the knowledge graph in the question-answering system so that it contains richer information, thereby improving the accuracy and practicality of the question-answering system.

[0171] ②Recommendation system:

[0172] Recommendations based on knowledge graphs: The DSET model can analyze user behaviors and entity relationships, and complete the relationships between users and recommended entities, thereby making more accurate recommendations, such as making recommendations based on user interests, preferences, and entity relationships in the knowledge graph.

[0173] ③Smart Search:

[0174] Intelligent search integrated with knowledge graph: The DSET model helps intelligent search engines better understand user search intent through semantic analysis and recognition, and provide more relevant search results.

[0175] References:

[0176] [1]Bordes A,Usunier N,Garcia-Duran A,et al.Translating embeddings formodeling multi-relational data[J].Advances in neural informationprocessingsystems,2013,26.

[0177] [2]Wang Z,Zhang J,Feng J,et al.Knowledge graph embedding bytranslating on hyperplanes[C] / / Proceedings of theAAAI conference onartificial intelligence.2014,28(1).

[0178] [3]Lin Y,Liu Z,Sun M,et al.Learning entity and relation embeddingsfor knowledge graph completion[C] / / Proceedings oftheAAAI conference onartificial intelligence.2015,29(1).

[0179] [4]Ji G,He S,Xu L,et al.Knowledge graph embedding via dynamic mappingmatrix[C] / / Proceedings ofthe 53rd annual meeting of the association forcomputational linguistics and the 7th international joint conference onnatural language processing(volume 1:Longpapers).2015:687-696.

[0180] [5]Sun Z,Deng Z H,Nie JY,et al.Rotate:Knowledge graph embeddingbyrelational rotation in complex space[J].arXiv preprint arXiv:1902.10197,2019.

[0181] [6]Nickel M,Tresp V,Kriegel H P.A three-way model for collectivelearning on multi-relational data[C] / / Icml.2011,11(10.5555):3104482.3104584.

[0182] [7]Yang B,Yih W,He X,et al.Embedding entities and relations forlearning and inference in knowledge bases[J].arXiv preprint arXiv:1412.6575,2014.

[0183] [8]Trouillon T,Welbl J,Riedel S,et al.Complex embeddings for simplelink prediction[C] / / International conference on machine learning.PMLR,2016:2071-2080.

[0184] [9]Kazemi S M,Poole D.Simple embedding for link prediction inknowledge graphs[J].Advances in neural information processing systems,2018,31.

[0185]

[10] Dettmers T,Minervini P,Stenetorp P,et al.Convolutional 2dknowledge graph embeddings[C] / / Proceedings of the AAAI conference onartificial intelligence.2018,32(1).

[0186]

[11] Nguyen D Q,Nguyen T D,Nguyen D Q,et al.A novel embedding modelfor knowledge base completion based on convolutional neuralnetwork[J].arXivpreprint arXiv:1712.02121,2017.

[0187]

[12] Vashishth S,Sanyal S,Nitin V,et al.Interacte:Improvingconvolution-based knowledge graph embeddings by increasing featureinteractions[C] / / Proceedings of the AAAI Conference on ArtificialIntelligence.Palo Alto,USA:AAAIPress,2020,34(03):3009-3016.

[0188]

[13] Schlichtkrull M,Kipf T N,Bloem P,et al.Modeling relational datawith graph convolutional networks[C] / / The Semantic Web:15th InternationalConference,ESWC 2018,Heraklion,Crete,Greece,June 3–7,2018,Proceedings15.Springer InternationalPublishing,2018:593-607.

[0189]

[14] Shang C, Tang Y, Huang J, et al. End-to-end structure-awareconvolutional networks for knowledge base completion[C] / / ProceedingsoftheAAAI conference on artificial intelligence.2019,33(01):3060-3067.

[0190]

[15] Nathani D, Chauhan J, Sharma C, et al.Learning attention-basedembeddings for relation prediction in knowledge graphs[J].arXivpreprintarXiv:1906.01195,2019.

[0191]

[16] Vashishth S,Sanyal S,Nitin V,et al.Composition-based multi-relational graph convolutional networks[J].arXiv preprint arXiv:1911.03082,2019.

[0192]

[17] Wang Zhiyue, Yu Qing, Wang Nan, et al. A review of intelligent question answering based on knowledge graph[J]. Computer Engineering and Applications, 2020, 56(23): 11. DOI: 10.3778 / j.issn.1002-8331.2004-0370.

[0193]

[18] Zhou Jing, Sun Ximin, Yu Xiaokun, et al. Knowledge graph and data application - intelligent recommendation[J]. Telecommunications Science, 2019, 35(8):8. DOI:CNKI:SUN:DXKX.0.2019-08-019.

[0194]

[19] Berant J,ChouA,Frostig R,et al.Semantic parsing on freebase fromquestion-answer pairs[C] / / Proceedings of the 2013conference on empiricalmethods in natural language processing.2013:1533-1544.

[0195]

[20] Mahdisoltani F,Biega J,Suchanek F.Yago3:A knowledge base frommultilingual wikipedias[C] / / 7th biennial conference on innovative datasystems research.CIDR Conference,2014.

[0196]

[21] Erxleben F,Günther M, M,et al.Introducing Wikidata to thelinked data web[C] / / International semantic web conference.Berlin,Germany:Springer,2014:50-65.

[0197]

[22] Miller GA.WordNet:a lexical database forEnglish[J].CommunicationsoftheACM,1995,38(11):39-41.

[0198]

[23] Bollacker K,Evans C,Paritosh P,et al.Freebase:a collaborativelycreated graph database for structuring human knowledge[C] / / Proceedings ofthe2008ACM SIGMOD international conference on Management ofdata.2008:1247-1250.

[0199]

[24] Boschee E, Lautenschlager J, O'Brien S, et al. ICEWS Coded Event Data [M / OL]. [2021-01-22]. https: / / doi.org / 10.7910 / DVN / 28075.

[0200]

[25] Dong

[0201]

[26] Zhang Wenhao, Xu Zhenshun, Liu Na, et al. A review of knowledge graph completion methods [J / OL]. Computer Engineering and Applications, 1-16 [2024-04-05]. http: / / kns.cnki.net / kcms / detail / 11.2127.TP.20240105.1412.010.html.

[0202]

[27] Toutanova K,Chen D.ObservedVersus Latent Features for KnowledgeBase andText Inference[C] / / The 3rd Workshop on Continuous Vector Space Modelsandtheir Compositionality.2015.

[0203]

[28] Chao L,He J,Wang T,et al.Pairre:Knowledge graph embeddings viapaired relation vectors[J].arXiv preprint arXiv:2011.03798,2020.

[0204]

[29] I,Allen C,Hospedales T M.Tucker:Tensor factorization forknowledge graph completion[J].arXiv preprint arXiv:1901.09590,2019.

[0205]

[30] Zhou Z,Wang C,Feng Y,et al.JointE:Jointly utilizing 1D and 2Dconvolution for knowledge graph embedding[J].Knowledge-Based Systems,2022,240:108100.

[0206]

[31] Zhang Z,Li Z,Liu H,et al.Multi-scaledynamic convolutional networkfor knowledge graph embedding[J].IEEE Transactions on Knowledge andDataEngineering,2020,34(5):2335-2347.

[0207]

[32] Xue Z,Zhang Z,Liu H,et al.Learning knowledge graph embedding withmulti-granularity relational augmentation network[J].Expert SystemswithApplications,2023,233:120953.

[0208]

[33] Deng W,Zhang Y,Yu H,et al.Knowledge graph embedding based ondynamic adaptive atrous convolution and attention mechanism forlinkprediction[J].Information Processing&Management,2024,61(3):103642.。

[0209] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.

Claims

1. A knowledge graph completion method based on deep semantics and entity category information extraction, characterized in that: An end-to-end neural network DSET model is proposed, and the entities at the missing positions in the predicted knowledge graph are output based on the DSET model.

2. The knowledge graph completion method based on deep semantics and entity category information extraction according to claim 1 is characterized in that: The DSET model includes: a type similarity graph attention encoder TS-GAT that focuses on modeling entity type information, and a deep semantic information three-dimensional convolution decoder DS-Conv3 that is responsible for extracting deep semantic features of entities and relationships.

3. The knowledge graph completion method based on deep semantics and entity category information extraction according to claim 2 is characterized in that: The type similarity graph attention encoder TS-GAT considers the intrinsic connection between entity types in equivalence relations, and identifies equivalence relations by calculating the similarity of positive and negative relation embeddings during the aggregation process, thereby giving higher association weights to entities of the same type; at the same time, TS-GAT also considers the diversity information between entities, and aggregates triple-level information with both entities and relations for the central entity.

4. The knowledge graph completion method based on deep semantics and entity category information extraction according to claim 2 is characterized in that: The deep semantic information three-dimensional convolution decoder DS-Conv3 uses Gaussian function mapping to capture the deep and complex semantics of entities and relations, and utilizes three-dimensional convolution operations to extract local features of entities and relations while capturing the complex interaction patterns between them.

5. The knowledge graph completion method based on deep semantics and entity category information extraction according to claim 2 is characterized in that: The type similarity graph attention encoder TS-GAT proposes a head-tail type attention mechanism, which determines whether it is an equivalent relationship and whether the head and tail entities belong to the same category by calculating the cosine similarity between the forward and reverse relationship embeddings corresponding to the relationship; the specific implementation is as follows: First, the calculation formula for the positive and negative relationship similarity is: The symbol · represents the dot product of two vectors, and ||r||2 and ||r'||2 represent the bi-norm of vectors r and r', respectively. Since the value range of cosine similarity is between -1 and 1, in order to facilitate the subsequent attention score calculation, the value range of positive and negative relationship similarity is controlled between 0 and 1; Next, calculate the attention score between the head and tail entities: Among them, α ij Representing entity e i and entity e j The type attention score between ij Represents entity e i and entity e j The embedding vector of the relationship between ij 'Represents entity e i and entity e j The embedding vector corresponding to the reverse relationship between It is entity e i The neighbor relationship set of Next, the embedding vectors of the neighbor entities are weighted according to the attention scores and updated; the passed messages are calculated and updated in the following way: in, Indicates that the neighbor entity is passed to entity e i Type information, Represents all and entity e i A collection of connected entities, It is the mapping matrix of type information.

6. The knowledge graph completion method based on deep semantics and entity category information extraction according to claim 5 is characterized in that: The type similarity graph attention encoder TS-GAT also adopts a diverse information transmission process based on the attention mechanism, so that the central entity can obtain discriminative information from different connected triplets, which is specifically implemented as follows: The entity similarity attention score is used to measure the similarity between the neighbor entity and the central entity. The calculation formula is as follows: Among them, β ij It is entity e i About Entity e j Through the relationship ij Entity similarity attention score of ; After calculating the entity similarity attention scores of each neighbor entity, TS-GAT will perform weighted aggregation of the information of each neighbor entity based on these scores. In the aggregation process, not only the neighbor entity embeddings but also the relationship embeddings between them are considered. The two embeddings are fused by Hadamard product to obtain an updated representation of the diversity information. The calculation formula is as follows: in, Represents the entity e i Diversity information, Represents all entities e i The connected fact set, ⊙ represents the Hadamard product, It is the mapping matrix of diversity information.

7. The knowledge graph completion method based on deep semantics and entity category information extraction according to claim 6 is characterized in that: The update mechanism of the type similarity graph attention encoder TS-GAT is defined as follows: in, is a learnable parameter matrix, σ is the nonlinear activation function ReLU, e' i is the updated entity embedding, which will be used as the input entity embedding for the next iteration until the entire model training process is completed.

8. The knowledge graph completion method based on deep semantics and entity category information extraction according to claim 2 is characterized in that: The deep semantic information three-dimensional convolution decoder DS-Conv3 designs a multi-dimensional Gaussian mapping method to map and transform the embedding of entities and relationships, as follows: The mapping function for the entity is composed of a series of Gaussian functions φ i (e) where each kernel function focuses on the input embedding and a specific center point c i The similarity between them; specifically, for the i-th kernel function, we get: where c i represents the i-th Gaussian center, which can be obtained by model parameterization learning, σ i is the corresponding standard deviation; the mapping function covers d Gaussian functions, forming a mapping function Φ(e): Φ(e)=[φ1(e),φ2(e),…,φ d (e)] Through the mapping function Φ, the head entity embedding vector e s is converted into a new feature vector Similarly, the relation embedding r is also transformed into a new feature vector r Φ :

9. The knowledge graph completion method based on deep semantics and entity category information extraction according to claim 8, characterized in that: The deep semantic information three-dimensional convolution decoder DS-Conv3 also depicts the high-dimensional complex semantics of entities and relationships and their interactive information through three-dimensional convolution operations, which is specifically implemented as follows: Given the original head entity and relation embedding e s , r, and the embedding vector obtained by Gaussian mapping r Φ , the feature input is constructed as follows: Among them, ρ represents the reshaping operation, which reshapes the embedded vector into a two-dimensional feature map with a height of h and a width of w. Stack represents the stacking operation, and axis is the dimension in which the stacking operation is performed. F has a dimension of h×w×5, which represents the feature input tensor. Then use a three-dimensional convolution kernel to calculate the feature input tensor: X = σ(Conv3D(F)) Among them, X represents the output feature vector after three-dimensional convolution extracts interactive features and local features, σ represents the nonlinear activation function ReLU, Conv3D represents the three-dimensional convolution operation, and the size of the convolution kernel is set to c×k h ×k w ×2, c represents the number of channels, k h , k w Represent the height and width of the convolution kernel respectively, and its depth dimension is 2; Finally, the feature map obtained after the convolutional layer is converted into a one-dimensional feature vector using the flattening operation vec(): Out = vec(X).

10. A knowledge graph completion system based on deep semantics and entity category information extraction, characterized in that: The method comprises a memory, a processor and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps as claimed in any one of claims 1 to 9 can be implemented.

Images