Global neighborhood sensing knowledge graph reasoning method and system

By introducing internumerical centrality and hierarchical attention networks into the knowledge graph inference method, the problem of insufficient efficiency and accuracy of the knowledge graph inference method in the existing technology in multi-hop inference and graph global information mining is solved, and the problem of insufficient efficiency and accuracy of the knowledge graph inference method in the existing technology is improved. The reasoning accuracy and interpretability of the knowledge graph.

CN118964631APending Publication Date: 2024-11-15HEFEI UNIV OF TECH
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410975004.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing knowledge graph inference methods are insufficient in computing efficiency and scalability when processing large-scale graph data, and it is difficult to make full use of complex structural information, resulting in the inefficient multi-hop inference and graph global information mining.

Method used

Using the knowledge graph inference method of global neighborhood perception, the hierarchical attention layer and the relational semantic attention layer are designed by introducing entity-level structural attention layer and relation-level semantic attention layer in the encoder and decoder, and using the median centrality as implicit structural information. The network captures local semantics and global structural information, enhancing the model's inference ability.

Benefits of technology

It improves the accuracy and interpretability of knowledge graph inference, optimizes the model's recognition ability for complex relationships and long-distance dependencies, and enhances the global understanding and reasoning ability of knowledge graphs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964631A_ABST
    Figure CN118964631A_ABST
Patent Text Reader

Abstract

The invention provides a global neighborhood sensing knowledge graph reasoning method and system, and relates to the technical field of knowledge graphs. According to the method, the knowledge graph is acquired, and related parameters in the knowledge graph are defined; and processing the defined knowledge graph based on a pre-constructed global neighborhood sensing knowledge graph inference model to obtain an inference result. Wherein the knowledge graph inference model comprises an encoder and a decoder, the encoder comprises an entity-level structure attention layer and a relation-level semantic attention layer, and betweenness centrality is used as a learning source of implicit structure information in the entity-level structure attention layer. According to the method, global structure information is considered, betweenness centrality is used as implicit structure information in complex reasoning, the problem that an existing method is insufficient in implicit information mining and utilization is solved, the cognitive ability of the model for existing knowledge is enhanced, and therefore the reasoning mechanism of the knowledge graph is optimized, and the reasoning accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph technology, and specifically to a global neighborhood-aware knowledge graph reasoning method and system. Background Art

[0002] As a key technology of artificial intelligence, knowledge graph not only provides a powerful representation tool for complex knowledge systems, but also is the key to promoting the development of intelligent systems towards cognitive intelligence. It describes the subject and its connection in the real world, describes the domain knowledge in the way humans perceive things, organizes the knowledge of the real world into descriptions of concepts and their relationships, and provides an effective knowledge representation tool for understandable artificial intelligence. With the automatic learning and construction of knowledge graphs and the increasing enrichment of knowledge, the cognitive ability of intelligent systems has been greatly improved, making them think in a similar way to humans, and providing a foundation for human-computer collaboration. Moreover, in the context of the research boom of Large Language Models (LLMs), by combining the rich entity relationships and reasoning mechanisms of knowledge graphs, it can not only help large models improve the accuracy of decision-making and reasoning processes, but also enhance the interpretability of reasoning by large models.

[0003] However, with the continuous evolution of intelligent system requirements and the increasing complexity of the real world, knowledge graphs are becoming increasingly large, with more and more subjects and more and more complex relationships between subjects. The knowledge graph construction process extracts entities and their relationships based on existing knowledge, resulting in large entities but sparse relationships in the knowledge graph. In fact, in addition to the explicit relationships directly described in the graph, many relationships can be derived through reasoning based on existing relationships. Knowledge reasoning aims to mine new information from existing knowledge, which is of great significance for enriching the content of knowledge graphs and improving their application value. Especially in the context of intelligent systems increasingly pursuing autonomous learning and decision-making, effective knowledge graph reasoning can not only help the system acquire new knowledge, but also enhance its ability to understand and handle complex problems. Figure 1 This paper describes the reasoning process of an intelligent case analysis knowledge graph in case detection. This reasoning process is based on an incomplete knowledge graph, that is, existing entities and the direct relationships between them (as shown by the solid line). Through a series of reasoning steps, the missing relationships in the graph can be inferred (as shown by the dotted line). For example, through the information of eyewitnesses, victims, and people who have recently contacted the suspect, a strong chain of evidence can be formed to find the most likely suspect. This case analysis based on knowledge graph reasoning can not only improve the efficiency of large and complex case investigations and enhance the relevance of evidence, but also help the police fully understand the case, assist decision-making, and promote the realization of judicial justice through intelligent and automated analysis processes.

[0004] At present, the methods of knowledge graph reasoning are mainly concentrated in three aspects: methods based on representation learning, methods based on neural networks, and methods based on graph neural networks. Among these existing technical methods, methods based on graph neural networks have achieved better reasoning effects due to their natural adaptability to graph structured data. This type of knowledge graph reasoning method based on graph neural networks can directly perform calculations and information transmission on the graph, effectively capture the multi-layer relationships between entities and the deep dependencies of the graph, and improve the accuracy, efficiency and interpretability of reasoning. With the development of the attention mechanism, scholars have successively explored and applied graph attention networks to knowledge graph reasoning tasks, achieving better reasoning effects than general graph neural networks. The attention mechanism enables GAT to not only capture the characteristics of neighbor nodes, but also dynamically adjust the aggregation strategy according to the importance of neighbor nodes, which is very beneficial to the information acquisition of entity nodes in knowledge graph reasoning.

[0005] Recently, in order to further mine and use effective information in knowledge graphs for reasoning, many researchers have designed reasoning models for knowledge graphs based on graph neural networks and centered around feature enhancement. These methods generally adopt the framework of encoders and decoders, and introduce more or finer-grained semantic information such as relationship semantics, relationship type, entity similarity, etc. into the encoder's graph attention network to obtain more interpretable and less noisy entity representations, thereby improving the accuracy of reasoning.

[0006] However, with the continuous expansion and deepening of application scenarios, the graph attention network-based reasoning models improved around feature enhancement are difficult to fully utilize the complex structural information in the knowledge graph, and the computational efficiency and scalability when processing large-scale graph data may be limited, which makes these methods not efficient or accurate enough in multi-hop reasoning and mining of global graph information. Summary of the invention

[0007] 1. Technical issues to be resolved

[0008] In view of the shortcomings of the prior art, the present invention provides a global neighborhood-aware knowledge graph reasoning method and system, which solves the technical problem of low accuracy of the existing knowledge graph reasoning methods.

[0009] (II) Technical solution

[0010] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0011] In a first aspect, the present invention provides a global neighborhood-aware knowledge graph reasoning method, comprising:

[0012] S1. Obtain the knowledge graph and define relevant parameters in the knowledge graph;

[0013] S2. Process the defined knowledge graph based on a pre-built global neighborhood-aware knowledge graph reasoning model to obtain an inference result; wherein, the knowledge graph reasoning model includes an encoder and a decoder, the encoder includes a first GAT layer and a second GAT layer, each GAT layer includes an entity-level structural attention layer and a relationship-level semantic attention layer, wherein in the entity-level structural attention layer, betweenness centrality is used as a learning source of implicit structural information.

[0014] In a second aspect, the present invention provides a global neighborhood-aware knowledge graph reasoning system, characterized in that an acquisition module is used to acquire a knowledge graph and define relevant parameters in the knowledge graph;

[0015] An inference module is used to process the defined knowledge graph based on a pre-built global neighborhood-aware knowledge graph inference model to obtain an inference result; wherein, the knowledge graph inference model includes an encoder and a decoder, the encoder includes a first GAT layer and a second GAT layer, each GAT layer includes an entity-level structural attention layer and a relationship-level semantic attention layer, wherein in the entity-level structural attention layer, betweenness centrality is used as a learning source of implicit structural information.

[0016] In a third aspect, the present invention provides a computer-readable storage medium, characterized in that it stores a computer program for a global neighborhood-aware knowledge graph reasoning method, wherein the computer program enables a computer to execute the global neighborhood-aware knowledge graph reasoning method as described above.

[0017] In a fourth aspect, the present invention provides an electronic device, comprising:

[0018] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include methods for executing the global neighborhood-aware knowledge graph reasoning method as described above.

[0019] (III) Beneficial effects

[0020] The present invention provides a global neighborhood-aware knowledge graph reasoning method and system. Compared with the prior art, it has the following beneficial effects:

[0021] The present invention proposes a method that considers global structural information and uses betweenness centrality as implicit structural information in complex reasoning to make up for the problem that existing methods are insufficient in mining and utilizing implicit information, enhance the model's ability to recognize existing knowledge, thereby optimizing the reasoning mechanism of the knowledge graph and improving the accuracy of reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 This is a schematic diagram of knowledge graph reasoning;

[0024] Figure 2 A block diagram of a global neighborhood-aware knowledge graph reasoning method according to an embodiment of the present invention;

[0025] Figure 3 This is the overall framework diagram of the global neighborhood-aware knowledge graph reasoning model;

[0026] Figure 4 This is a framework diagram of the encoder in the knowledge graph reasoning model;

[0027] Figure 5 It is the feature aggregation graph in the entity-level structure attention layer in the encoder;

[0028] Figure 6 Schematic diagram of case reasoning in the verification experiment;

[0029] Figure 7a , 7b To verify the ablation experimental results of implicit structural information in the experiment;

[0030] Figure 8a , 8b To verify the ablation results of the hierarchical semantic relationships in the experiment;

[0031] Figure 9 shows the ablation experiment results of the multi-head attention mechanism in the verification experiment. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] The embodiments of the present application solve the technical problem of low accuracy of existing knowledge graph reasoning methods by providing a global neighborhood-aware knowledge graph reasoning method and system, and realize the use of betweenness centrality as implicit structural information in complex reasoning to make up for the problem that existing methods are insufficient in mining and utilizing implicit information.

[0034] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0035] The main purpose of reasoning research on knowledge graphs is to discover the reasoning mechanism of unknown relations. At present, the most cutting-edge reasoning research on knowledge graphs is the reasoning model based on graph attention network improved around the direction of feature enhancement. However, with the continuous expansion and deepening of application scenarios, such methods are still not sufficient in the identification of complex relations and the mining of key structural information in knowledge graphs. For example, if the relationship types in the knowledge graph have been subdivided to the greatest extent, then the model's ability to understand the graph cannot be further enhanced through type-aware methods. Entity similarity is an important feature that connects entities, so are there other features that can represent the richer features of entities in knowledge reasoning? Local semantic information acquisition is necessary, but deeper mining of more implicit information in the graph from a structural perspective can enable the model to deepen its global understanding of the knowledge graph and complete the structural information that is crucial in knowledge reasoning.

[0036] The embodiments of the present invention are intended to overcome the limitations of existing knowledge graph reasoning technology in multi-hop relationship reasoning, especially the accuracy and efficiency problems when dealing with complex relationships and long-distance dependencies. Considering that there is still a large amount of unmined complex and useful information in the knowledge graph, it can better assist the model in identifying complex relationships and analyzing graph structures. Not only semantic information, the knowledge graph itself contains rich implicit structural information, which is crucial to understanding the global structure of the entire knowledge graph and guiding deep reasoning. For example, the degree centrality of a node reflects the activity of the entity; the betweenness centrality reflects the bridge role of the node in the graph network; the proximity centrality reflects the speed at which the node reaches other parts of the network; the clustering coefficient reflects the cluster structure in the network. Considering that intelligent systems need to understand not only individual entities or relationships when making complex decisions, but also grasp their status and role in the entire knowledge graph.

[0037] Therefore, in order to further meet the complex needs of intelligent systems for knowledge understanding and application, the embodiments of the present invention propose a global neighborhood-aware knowledge graph reasoning method and system, which aims to use advanced reasoning mechanisms to more accurately complete the reasoning task of the knowledge graph based on more comprehensive and higher-quality information. Specifically, the main purposes of the embodiments of the present invention include:

[0038] 1. Improve the accuracy of multi-hop reasoning: Propose a method that considers global structural information and uses betweenness centrality as implicit structural information in complex reasoning to make up for the problem that existing methods are insufficient in mining and utilizing implicit information, enhance the model's ability to recognize existing knowledge, and thus optimize the reasoning mechanism of the knowledge graph.

[0039] 2. Enhance the interpretability of the reasoning model: A global neighborhood-aware knowledge graph reasoning model (GAGAT) is designed. GAGAT guides the reasoning of knowledge graphs from a global perspective, using a hierarchical attention network to simultaneously consider the local semantic information and implicit global structural information of heterogeneous relations in the knowledge graph, and adds this information as features to the calculation of attention weights, thereby more effectively aggregating neighborhood entity features and improving the interpretability of knowledge graph reasoning.

[0040] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0041] The embodiment of the present invention provides a global neighborhood-aware knowledge graph reasoning method, such as Figure 2 As shown, including:

[0042] S1. Obtain the knowledge graph and define relevant parameters in the knowledge graph;

[0043] S2. Process the defined knowledge graph based on a pre-built global neighborhood-aware knowledge graph reasoning model to obtain an inference result; wherein, the knowledge graph reasoning model includes an encoder and a decoder, the encoder includes a first GAT layer and a second GAT layer, each GAT layer includes an entity-level structural attention layer and a relationship-level semantic attention layer, wherein in the entity-level structural attention layer, betweenness centrality is used as a learning source of implicit structural information.

[0044] The embodiment of the present invention proposes a method that considers global structural information, and uses betweenness centrality as implicit structural information in complex reasoning to make up for the problem that existing methods are insufficient in mining and utilizing implicit information, enhance the model's ability to recognize existing knowledge, thereby optimizing the reasoning mechanism of the knowledge graph and improving the accuracy of reasoning.

[0045] The following is a detailed description of each step:

[0046] In one embodiment, S1, obtain a knowledge graph and define relevant parameters in the knowledge graph. Specifically, it includes:

[0047] Knowledge Graph It is considered as a directed multi-relation graph, where E represents the set of entities, R represents the set of relations, and G represents the set of all triples in the knowledge graph. h, t∈E represent the head entity and tail entity of a given triple, r∈R represents the relation connecting entities h and t, and G′ represents the set of triples that do not exist in the knowledge graph; N i represents the set of neighborhood entities of entity i, BC represents the betweenness centrality of the entity, Φ represents the set of relationship paths, represents the relationship path, and η represents the set of shortest paths between entity pairs.

[0048] In one embodiment, S2, based on a pre-built global neighborhood-aware knowledge graph reasoning model, the defined knowledge graph is processed to obtain a reasoning result. The knowledge graph reasoning model includes an encoder and a decoder, the encoder includes an entity-level structural attention layer and a relationship-level semantic attention layer, and the betweenness centrality is used as a learning source of implicit structural information in the entity-level structural attention layer. Specifically, it includes:

[0049] like Figure 3 As shown in Figure 2, the pre-built global neighborhood-aware knowledge graph reasoning model (GAGAT) is an encoder-decoder framework. Figure 4 As shown in the figure, a hierarchical attention network is designed, which includes two parts: entity-level structural attention layer and relation-level semantic attention layer. The entity-level structural attention layer focuses on capturing the structural characteristics of neighborhood entities, and uses the hidden structural information betweenness centrality to provide the model with a global perspective to understand the importance of nodes in the network; the relation-level semantic attention layer is used to distinguish the contribution of different neighborhood relational semantics to the central entity. Through the hierarchical attention network, the model can effectively distinguish different relational semantics in the neighborhood, as well as different entities under the same relational semantics, and can effectively perceive the position and role of entity nodes in the entire knowledge graph. This method uses relational semantics to increase the effective knowledge content for reasoning, and uses implicit structural information to enhance the model's ability to discriminate the importance of knowledge information, effectively balancing the information aggregation of local semantics and global structure, making the representation of entities more comprehensive and accurate, while maintaining the simplicity and interpretability of the model.

[0050] Closeness centrality mainly measures the distance between a node and other nodes, but it ignores the degree of intermediate connections between nodes. In a knowledge graph, some nodes may be close to other nodes, but they may not be important in terms of information dissemination and influence dissemination. Therefore, closeness centrality may not reflect the structural role of the node well. The clustering coefficient measures the degree of connection between node neighbors, but it only considers the local connection of the node and cannot fully reflect the importance of the node in the entire graph. For example, some nodes may have a high clustering coefficient, but they do not have a great influence in the entire graph. Therefore, the clustering coefficient may not capture the structural role of the node well. Degree centrality measures the number of connections between nodes, but it ignores the specific connections between nodes. For example, some nodes may have a high degree centrality, but their connections may be mainly concentrated on certain specific nodes, which has little impact on the structure of the entire graph. Therefore, degree centrality may not reflect the structural role of the node well. The betweenness centrality measures the frequency of occurrence of a node in the shortest path between all pairs of nodes, reflecting the role of the node as a bridge in the network, enabling the model to consider the role of the node entity in a wider network structure, helping to understand how nodes influence or are influenced by remote nodes, and is a kind of hidden structural information of the graph that is crucial for link prediction tasks. In summary, although closeness centrality, clustering coefficient, and degree centrality may be useful in some cases, they are not always suitable for reasoning about knowledge graphs as entity features. Therefore, the embodiment of the present invention uses betweenness centrality as a learning source of implicit structural information to calculate entity-level structural attention.

[0051] The data processing process of the entity-level structure attention layer includes:

[0052] The embedding vectors of the central entity and the neighborhood entity are mapped to the same vector space for dimension transformation to obtain the projection features of the central entity and the projection features of the neighborhood entity. The expression is as follows:

[0053] h i ′=h i W1,h j ′=h j W1#(1)

[0054] in, It is a shared learnable parameter that maps the embedding vectors of different neighborhood entities to the same vector space for dimensional transformation. d represents the dimension of the input feature vector, and d′ represents the dimension of the output feature vector. are the original features of the central entity i and the neighborhood entity j, h i ′ and h j ′ is the projection feature of entities i and j.

[0055] The projection features of the central entity, the projection features of the neighboring entities, and the betweenness centrality information of the neighboring entities are concatenated, and the attention score e of the entity-level structural attention layer is obtained by learning the self-attention mechanism. ij . It is expressed as follows:

[0056]

[0057] in,

[0058]

[0059] δ st Represents the set of all shortest paths between entity s and entity t in the knowledge graph; η st (j) represents the number of paths passing through entity j in the shortest path set; [·||·||·] represents the concatenation operation; A shared learnable parameter that maps the concatenated high-dimensional features to real numbers, resulting in the attention score e ij .

[0060] Then, the softmax function is used to normalize the attention scores to obtain the attention weights It is expressed as follows:

[0061]

[0062] Among them, LeakyReLU(·) is a nonlinear activation function.

[0063] The feature aggregation process in the entity-level structure attention layer is as follows: Figure 4 shown.

[0064] According to the calculated attention weight α ij , linearly combine the embeddings of all neighbors of entity i by weighted features to obtain a new embedding representation of the central entity i, which is aggregated as follows:

[0065]

[0066] in, is the output of the entity-level structural attention layer, which represents the new features of entity i after aggregating neighborhood information under a specific semantic relationship path. σ(·) is the activation function. are learnable parameters.

[0067] Since this layer considers the contribution of the neighborhood entities of a single relationship path to the central entity, it can only capture one type of relationship semantic information. In order to further enhance the robustness of the model and learn more neighborhood information, multi-head attention is evolved and enhanced, and represented as a new embedding of specific relationship semantics, as shown below:

[0068]

[0069] Where K is the number of heads of the attention mechanism, h i ′(K) is the output representation of entity i of the K-th head. Different from the general GAT, the representations of multiple heads are averaged and calculated as follows:

[0070]

[0071] For different relationship paths, set a relationship path set When the entity node passes through the entity-level structure attention layer, it will output the new embedding of m groups of entity i

[0072] A single-layer GAGAT can obtain the one-hop neighborhood of an entity in one training iteration. When the number of layers and iterations is increased, the proposed model can obtain more effective information from the multi-hop neighborhood, thereby improving the reasoning ability of the model. After obtaining the representation of the knowledge graph containing rich neighborhood information, the global dependency between input and output needs to be modeled to fully learn the neighborhood features for the next stage of reasoning.

[0073] The data processing process in the relation-level semantic attention layer includes:

[0074] In addition to the structural information of neighborhood entities, the semantic information contained in different relationships in heterogeneous graphs must also be considered to make the embedding of entity nodes more comprehensive and detailed. In order to address the challenge of semantic information fusion, inspired by the heterogeneous graph neural network HAN, the embodiment of the present invention designs a relation-level semantic attention layer, so that the model can learn entity information of different triples according to different relationships, and aggregate different entities under different relationships and the same entity information under different relationships in a more fine-grained manner.

[0075] The m groups of node embeddings with specific relationship semantics obtained from the entity-level structural attention layer are used as the input of the relationship-level semantic attention layer to further learn the contribution of different semantic relationships to the central entity node. To express the Attention to relational semantics.

[0076] To obtain the importance of each relation path, the embodiment of the present invention transforms the specific embedding through a nonlinear transformation. The importance semantic specific embedding is then measured as the similarity between the transformed embedding and the semantic level attention vector q. In addition, the embodiment of the present invention averages the importance of all semantic specific node embeddings, which can be interpreted as the importance of each relation path. The attention weight of each relation path The calculation is as follows:

[0077]

[0078] Among them, W is the weight matrix, b is the bias vector, q is the relation-level attention vector, and entity embeddings in different relations and specific semantics will share all the above parameters.

[0079] After obtaining the importance of each relationship to the central entity, the embodiment of the present invention performs normalization through softmax to obtain the attention weight at the relationship level.

[0080]

[0081] Then, the original semantically specific entity embeddings are fused at the relation level using weighted features to obtain the final embedding h of the entity in the first layer GAT (Graph Attention Network) i ″, after multiple layers of GAT, a new entity embedding h i ″′.

[0082]

[0083] The decoder adopts ConvKB, which has good decoding performance in the GAT-based coding model.

[0084] ConvKB re-expresses each triple as a hash matrix, where each column corresponds to an element (entity or relationship) of the triple. By introducing the convolution operation, ConvKB uses multiple filters of the same shape in the convolution layer, so that the local relationship between different elements in the triple can be captured. The extraction of this local information provides the model with more fine-grained cognition, which helps to better understand the complex patterns in the knowledge graph. After the convolution layer, ConvKB connects the multiple feature maps generated to form a compact feature vector, which is regarded as an abstract representation of the corresponding input triple. This representation is fused with the weight vector through a dot product operation to produce the final score. The score function of ConvKB is designed to comprehensively consider the interaction of each element in the triple, thereby providing more comprehensive information for judging the effectiveness of the triple. The score function is specifically defined as follows:

[0085] f c (h,r,t)=concat(σ([h,r,t]*Ω))·w#(11)

[0086] Among them, * represents the convolution operation, is the parameter of the linear transformation matrix, σ is the activation function of ReLU. concat represents the concatenation operator.

[0087] ConvKB is trained by minimizing the following loss function using the Adam optimizer.

[0088]

[0089] Among them, l (h,r,t) is a label indicating whether the triple (h, r, t) is valid. l (h,r,t) The value of is 1, and for other triples l (h,r,t) The value of is 0.

[0090] The following comparative experiments are used to verify the effectiveness and advantages of the GAGAT model:

[0091] 1. Dataset and Experimental Setup

[0092] Two commonly used benchmark datasets are used: FB15K-237 and WN18RR.

[0093] FB15K-237: is a subset of FB15K, which is a widely used dataset of the knowledge graph Freebase. FB15K-237 is a variant of FB15K that removes the inverse relations in FB15K. The "15K" indicates that the knowledge base contains about 15,000 subject entities, and the "237" indicates that there are 237 relations.

[0094] WN18RR: It is a subset of WN18, which is a dataset of the knowledge graph WordNet. Similar to FB15K-237, WN18RR deletes the inverse relations in WN18 and was first created and used in the paper of the ConvE model.

[0095] Table 1 describes the number of entity relations in the two datasets. Each dataset is divided into three groups: training set, validation set and test set.

[0096] Table 1 Dataset details

[0097]

[0098]

[0099] In the experimental parameter settings, the input feature dimension of entities and relations is 50, the output feature dimension is 128, and the number of attention heads is 8. In order to optimize the training process, the Adam optimizer is used and the batch size is set to 16. TransE is used as a pre-trained model to initialize the embedding of entities and relations as the input of the encoder. For different datasets, it is found that the following settings of some parameters can obtain better results. For the FB15k-237 dataset, the learning rate is set to 0.0009, the dropout is set to 0.04, and the L2 regularization is set to 0.00008. For the WN18RR dataset, the learning rate is set to 0.0003, the dropout is set to 0.02, and the L2 regularization is set to 0.00014. In addition, the weight initialization of the model uses the Kaiming method to adapt to the LeakyReLU activation function used in the model.

[0100] The negative sampling strategy is to obtain negative samples by randomly replacing the head entity or the tail entity. To prevent the model from overfitting too early and to ensure that the model's performance on the validation set is optimal, an early stopping strategy is implemented. The early stopping trigger condition is that the validation set performance has no significant improvement in 20 consecutive training cycles.

[0101] 2. Evaluation Metrics and Baselines

[0102] Four evaluation indicators are used in the experiment to measure the performance of the model, namely, mean reciprocal ranking (MRR) and the average proportion of correct entities in the top k entities (Hits@10, Hits@3, Hits@1). They are commonly used indicators for evaluating the performance of knowledge graph reasoning models, and are mainly used to measure the accuracy and efficiency of the model in predicting entity relationships in knowledge graphs. The specific calculation logic is as follows:

[0103]

[0104] |N t | is the set of triples N t The number of ranks i refers to the link prediction rank of the i-th triplet.

[0105] Hits@k (k=1, 3, 10) is calculated as follows:

[0106]

[0107] in, is an indicator function which equals 1 if the condition is true, otherwise 0.

[0108] For each evaluation metric, a higher score indicates better performance.

[0109] The principle of selecting baseline models is to use the reasoning models of classic knowledge graphs with superior performance and the currently advanced models in the field of graph attention network feature enhancement as baselines, including the representative methods DisMult and Complex in the representation learning-based method class; the excellent methods ConvE and ConvKB based on neural networks; the methods R-GCN, CompGCN, and SACN based on graph neural networks, as well as some further excellent model methods TAGAT and SD-GAT based on graph attention networks.

[0110] 3. Link prediction task results

[0111] Table 2 shows the evaluation index results of the baseline model and the model of the embodiment of the present invention on the two datasets FB15k-237 and WN18RR. The results of the DisMult and Complex models are taken from the ConvE paper (since the original paper did not use these two datasets, the results on the FB15k-237 and WN18RR datasets were obtained for the first time in ConvE), and the results of other baseline models are all taken from the original paper.

[0112] Table 2 Experimental results on WN18RR and FB15K-237 datasets

[0113]

[0114]

[0115] In general, GAGAT obtains better scores than all the baselines, demonstrating the superiority of our model. Specifically, in the experimental results of the FB15k-237 dataset, GAGAT achieves an improvement of 0.01 in the three indicators of MRR, Hits@10, and Hits@3, and has the same evaluation results as the advanced model SD-GAT in the indicator of Hits@1. In the experimental results of the WN18RR dataset, GAGAT has an improvement of 0.03 in the MRR indicator, 0.02 in the Hits@10 indicator, and 0.01 in the Hits@3 indicator, and is slightly lower than the SD-GAT model in Hits@1. The slightly poor performance of Hits@1 may be due to the low complexity of the relationships in the WN18RR dataset, and the more dispersed connections of the graph, so the performance of betweenness centrality is difficult to highlight.

[0116] In addition to verifying the results of the method of the present invention on the link prediction task on the dataset, examples are used to further illustrate the progress of the method proposed in the present invention in multi-hop reasoning. Figure 6 As shown, in the process of reasoning about the criminal in a case, the reasoning process is to build a chain of evidence through a series of intermediate steps and related information to support the conclusion that a suspect is the criminal.

[0117] For example, in the one-hop reasoning of preliminary screening of suspects, under the same suspect relationship, the entity features of the tail entity tend to be used to evaluate the potential association with the case. In this round of reasoning, the betweenness centrality of suspect A is calculated to be 0.176, and the betweenness centrality of suspect B is 0.306, which means that suspect B plays a key role in multiple related information in the case and becomes a potential bridge. Therefore, suspect B is selected as a key link in the chain of evidence for further in-depth reasoning. In the subsequent second-hop reasoning, each relationship associated with suspect B is only connected to one entity, so there are multiple relationship choices related to suspect B. In order to determine the best reasoning path, the method of the embodiment of the present invention weighs the importance of different relationships. The relationship of "recent contact" has significant significance in criminal investigation, so it is selected to infer people who have had recent contact with suspect B. The direct connection between suspect B and victim E is found, which provides new clues for solving the case. In the third-hop reasoning, the relationship between suspect B and victim E is further explored. Through in-depth analysis, the relationship between them as colleagues was revealed. This discovery not only enhanced the integrity of the chain of evidence, but also increased the suspicion of suspect B. Combining the reasoning results of the first three hops, a strong chain of evidence pointing to suspect B was successfully constructed. This chain of evidence not only covers multiple key related information, but also gradually reveals the truth of the case through logically rigorous reasoning steps. Therefore, based on the above analysis, it is reasonable to believe that suspect B is the most likely criminal in case X. In the process of constructing a chain of evidence pointing to the criminal, the method of the embodiment of the present invention relies on capturing the characteristics of entities in network relationships, and also pays attention to the importance of different relationships in case analysis. The method of the embodiment of the present invention not only ensures the coherence of reasoning, but also enhances the persuasiveness of the chain of evidence.

[0118] This experiment fully demonstrates the unique advantages of the method of the embodiment of the present invention in multi-hop reasoning. It can not only effectively integrate and analyze the existing direct relationships in the knowledge graph, but also reveal the key clues hidden in the complex relationships through intelligent reasoning.

[0119] 4. Ablation experiment

[0120] In order to demonstrate the effectiveness of each component in the model of the present invention, key components in the model, such as implicit structural information, the hierarchical attention structure considering relational semantic information, and the multi-head attention mechanism, are removed one by one to evaluate their respective importance and contribution to the model performance.

[0121] (1) Ablation experiment of implicit structural information:

[0122] In order to verify the effectiveness of implicit information modeling, ablation experiments were conducted on both datasets. A model NIGAT (None Implicit Information GraphAttention Network Model) that did not utilize implicit information betweenness centrality was constructed for comparison with GAGAT. The settings of other specific parameters in the experiment remained unchanged.

[0123] The experimental comparison results are as follows Figure 7a , 7b As shown in the figure, whether on the FB15k-237 dataset or the WN18RR dataset, the model GAGAT of the embodiment of the present invention performs better than NIGAT in terms of MRR, Hits@10, Hits@3, and Hits@1. This shows that implicit structural information plays an important role in capturing the feature information of entities and capturing deep semantic relationships, verifying the effectiveness of modeling using implicit structural information.

[0124] (2) Ablation Experiment of Relational Semantic Information

[0125] In order to verify the effectiveness of the hierarchical attention structure for the use of relational semantic information, the method of the present invention compares two different graph attention model structures: one is a model that removes the framework of relational semantics and only considers the information brought by adjacent tail entities and betweenness centrality for reasoning (NRGAT). The other is a model that considers the hierarchical graph attention network structure and relational semantic information (GAGAT).

[0126] like Figure 8a , 8b As shown in the experimental results, the hierarchical graph attention model (GAGAT) performs better than the single-layer model (NRGAT) that only considers adjacent tail entities and betweenness centrality. Specifically, on the FB15k-237 dataset, GAGAT performs better than NRGAT in all four indicators, and on the WN18RR dataset, GAGAT also performs better than NRGAT. This shows that considering relational semantics can enable the model to understand the relational information in the graph more comprehensively. By incorporating relational semantics into the model, the model can better distinguish the contribution of different relations to the central entity, thereby improving the model's ability to understand complex relations in the knowledge graph. Secondly, the use of a hierarchical graph attention network structure also plays a key role. This structure allows the model to effectively distinguish entity features under different relational semantics during the reasoning process, while considering local semantic information and global structural information, better perceive the position and role of entity nodes in the entire knowledge graph, and thus better aggregate the features of neighborhood entities. Such a hierarchical design makes the model's feature representation more comprehensive and interpretable, thereby improving the model's performance in knowledge graph reasoning tasks.

[0127] (3) Ablation Experiment of Multi-Head Attention Mechanism

[0128] In order to verify the effect of the multi-head attention mechanism in GAGAT, single-head attention, dual-head attention, four-head attention, eight-head attention and sixteen-head attention are set respectively, and further experiments are carried out on two data sets, and other parameter settings remain unchanged. According to the method proposed in the embodiment of the present invention, appropriately setting multi-head attention can enhance the feature expression ability for entities. Because each head can focus on different aspects of the input, it helps the model learn more feature information.

[0129] The experimental results are as follows Figure 9a , 9b As shown in the figure, the experimental results on the two data sets have the same trend. In the process of increasing from single-head attention to eight-head attention, the performance of the model improves with the increase of attention heads. When the model adopts sixteen-head attention, the performance of the model does not improve significantly but decreases. According to the experimental results, it can be analyzed that properly setting multi-head attention can enhance the feature expression ability of entities. Because each head can focus on different aspects of the input, it helps the model learn more feature information. However, after the number of heads increases to a certain extent, the performance improvement will gradually weaken or even lead to performance degradation. This may be because a larger number of heads increases the computational complexity and may cause overfitting, which leads to a decrease in the performance of the indicator. This shows that a reasonable setting of the number of heads of the multi-head attention mechanism can effectively improve GAGAT's ability to capture feature information, but too large a number of attention layer heads cannot further improve the model's perception ability.

[0130] The embodiment of the present invention also provides a global neighborhood-aware knowledge graph reasoning system, including:

[0131] The acquisition module is used to obtain the knowledge graph and define the relevant parameters in the knowledge graph;

[0132] An inference module is used to process the defined knowledge graph based on a pre-built global neighborhood-aware knowledge graph inference model to obtain an inference result; wherein, the knowledge graph inference model includes an encoder and a decoder, the encoder includes a first GAT layer and a second GAT layer, each GAT layer includes an entity-level structural attention layer and a relationship-level semantic attention layer, wherein in the entity-level structural attention layer, betweenness centrality is used as a learning source of implicit structural information.

[0133] It can be understood that the global neighborhood-aware knowledge graph reasoning system provided by the embodiment of the present invention corresponds to the above-mentioned global neighborhood-aware knowledge graph reasoning method, and the explanations, examples, beneficial effects, etc. of the relevant contents can refer to the corresponding contents in the global neighborhood-aware knowledge graph reasoning method, which will not be repeated here.

[0134] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program for global neighborhood-aware knowledge graph reasoning, wherein the computer program enables a computer to execute the global neighborhood-aware knowledge graph reasoning method as described above.

[0135] An embodiment of the present invention also provides an electronic device, comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the global neighborhood-aware knowledge graph reasoning method as described above.

[0136] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0137] 1. The embodiment of the present invention proposes a method that considers global structural information, and uses betweenness centrality as implicit structural information in complex reasoning to make up for the problem that existing methods are insufficient in mining and utilizing implicit information, enhance the model's ability to recognize existing knowledge, thereby optimizing the reasoning mechanism of the knowledge graph and improving the accuracy of reasoning.

[0138] 2. The embodiment of the present invention designs a global neighborhood-aware knowledge graph reasoning model (GAGAT). GAGAT guides the reasoning of knowledge graphs from a global perspective, using a hierarchical attention network to simultaneously consider the local semantic information and implicit global structural information of heterogeneous relationships in the knowledge graph, and adds this information as features to the calculation of attention weights, thereby more effectively aggregating neighborhood entity features and improving the interpretability of knowledge graph reasoning.

[0139] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0140] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A global neighborhood-aware knowledge graph reasoning method, It is characterized in that include: S1. Obtain the knowledge graph and define relevant parameters in the knowledge graph; S2. Process the defined knowledge graph based on a pre-built global neighborhood-aware knowledge graph reasoning model to obtain an inference result; wherein, the knowledge graph reasoning model includes an encoder and a decoder, the encoder includes a first GAT layer and a second GAT layer, the GAT layer includes an entity-level structural attention layer and a relationship-level semantic attention layer, wherein in the entity-level structural attention layer, betweenness centrality is used as a learning source of implicit structural information.

2. The global neighborhood-aware knowledge graph reasoning method according to claim 1, It is characterized in that The relevant parameters in the definition knowledge graph include: Knowledge Graph It is considered as a directed multi-relation graph, where E represents the set of entities, R represents the set of relations, and G represents the set of all triples in the knowledge graph; h, t∈E represent the head entity and tail entity of a given triple, r∈R represents the relationship connecting entities h and t, and G′ represents the set of triples that do not exist in the knowledge graph; N i represents the neighborhood entity set of entity i, BC represents the betweenness centrality of the entity, Φ represents the relationship path set, represents the relationship path, and η represents the set of shortest paths between entity pairs.

3. The global neighborhood-aware knowledge graph reasoning method according to claim 1, It is characterized in that The data processing process in the entity-level structure attention layer includes: The attention weight α of the entity-level structural attention layer is obtained based on the central entity, neighboring entity, and the betweenness centrality information of the neighboring entity. ij ; According to the calculated attention weight α ij , linearly combine the embeddings of all neighbors of entity i by weighted features to obtain the new embedding representation of the central entity i The new embedding representation of the central entity is evolved and enhanced through multi-head attention, and the multi-head representations are averaged to output the new embedding of m groups of entity i.

4. The global neighborhood-aware knowledge graph reasoning method according to claim 3, It is characterized in that The attention weight α of the entity-level structural attention layer is obtained based on the central entity, the neighborhood entity and the betweenness centrality information of the neighborhood entity ij ,include: The embedded vectors of the central entity and the neighborhood entity are mapped to the same vector space for dimension transformation to obtain the projection features of the central entity and the projection features of the neighborhood entity; the expression is as follows: h i ′=h i W 1 ,h j ′=h j W 1 in, It is a shared learnable parameter that maps the embedding vectors of different neighborhood entities to the same vector space for dimensional transformation. d represents the dimension of the input feature vector, and d′ represents the dimension of the output feature vector. are the original features of the central entity i and the neighborhood entity j, h i ′ and h j ′ is the projection feature of entities i and j; The projection features of the central entity, the projection features of the neighboring entities, and the betweenness centrality information of the neighboring entities are concatenated, and the attention score e of the entity-level structural attention layer is obtained by learning the self-attention mechanism. ij ; Its expression is as follows: in, δ st Represents the set of all shortest paths between entity s and entity t in the knowledge graph; η st (j) represents the number of paths passing through entity j in the shortest path set; [·||·||·] represents the concatenation operation; A shared learnable parameter that maps the concatenated high-dimensional features to real numbers, resulting in the attention score e ij ; Use the softmax function to normalize the attention score to get the attention weight It is expressed as follows: Among them, LeakyReLU(·) is a nonlinear activation function.

5. The global neighborhood-aware knowledge graph reasoning method according to claim 1, It is characterized in that The data processing process of the relation-level semantic attention layer includes: The new embedding of the m groups of entities i output by the entity-level structure attention layer The importance of each relationship path is averaged. The calculation is as follows: Where W is the weight matrix, b is the bias vector, and q is the relation-level attention vector; After obtaining the importance of each relationship to the central entity, normalize it through softmax to get the relationship-level attention weight: The original semantically specific entity embeddings are fused at the relation level using weighted features to obtain the final embedding h of the entity in the first layer of GAT. i ″:

6. The global neighborhood-aware knowledge graph reasoning method according to any one of claims 1 to 5, It is characterized in that The decoder includes a ConvKB.

7. The global neighborhood-aware knowledge graph reasoning method according to claim 6, It is characterized in that ConvKB is trained by minimizing the following loss function using the Adam optimizer, calculated as follows: Among them, l (h,r,t) is a label indicating whether the triple (h, r, t) is valid. The value of is 1, and for other triples The value of is 0.

8. A global neighborhood-aware knowledge graph reasoning system, It is characterized in that include: The acquisition module is used to obtain the knowledge graph and define the relevant parameters in the knowledge graph; An inference module is used to process the defined knowledge graph based on a pre-built global neighborhood-aware knowledge graph inference model to obtain an inference result; wherein, the knowledge graph inference model includes an encoder and a decoder, the encoder includes a first GAT layer and a second GAT layer, each GAT layer includes an entity-level structural attention layer and a relationship-level semantic attention layer, wherein in the entity-level structural attention layer, betweenness centrality is used as a learning source of implicit structural information.

9. A computer-readable storage medium, It is characterized in that It stores a computer program for a global neighborhood-aware knowledge graph reasoning method, wherein the computer program enables a computer to execute the global neighborhood-aware knowledge graph reasoning method as described in any one of claims 1 to 7.

10. An electronic device, It is characterized in that include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the global neighborhood-aware knowledge graph reasoning method as described in any one of claims 1 to 7.

Citation Information

Cited By

  • Knowledge graph complementary construction method based on implicit neighbor recognition and global information perception

    CN120781943A

  • Knowledge graph completion method based on implicit neighbor identification and global information perception

    CN120781943B