Attention perception path reasoning method of knowledge graph

Abstract

The invention discloses an attention perception path reasoning method for a knowledge graph, and the method comprises the steps: firstly, generating an entity representation containing local semantics through employing a graph attention network coding entity and an adjacency relation, and synchronously obtaining an attention weight representing the association intensity between entities; secondly, innovatively providing a target-guided biased random walk path sampling strategy, and adaptively exploring a high-quality multi-hop semantic path related to a target task by taking the attention weight as a bias; then, information aggregation is carried out on the sampled semantic paths through a path encoder, and global path representation is obtained; and finally, carrying out deep fusion on the local entity representation and the global path representation, and jointly inputting the local entity representation and the global path representation into a prediction layer to carry out knowledge graph link prediction. According to the method, local structure perception and global path reasoning are cooperatively optimized through an attention mechanism, so that the link prediction precision and interpretability of the knowledge graph are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and knowledge graph technology, specifically relating to an attention-aware path reasoning method for knowledge graphs. Background Technology

[0002] In the field of knowledge graph construction and reasoning, achieving efficient intelligent decision support is one of its core technological challenges. Currently, knowledge graph reasoning and solution generation often rely on expert experience, facing problems such as opaque processes, limited efficiency, and difficulty in converting implicit knowledge. Furthermore, the rich entity relationships and complex logical paths inherent in unstructured text have not been fully explored, resulting in insufficient automated reasoning capabilities in knowledge service systems.

[0003] Existing knowledge graph reasoning methods have significant limitations when dealing with complex domains: First, traditional path sampling strategies based on random walks lack directionality, making it difficult to effectively filter out high-quality semantic paths related to specific queries from large-scale graph structures, resulting in low reasoning efficiency and weak interpretability; Second, most methods treat local neighborhood information and global path information in isolation or simply combine them, lacking a dynamic collaborative fusion mechanism, and are unable to accurately model the interaction between local evidence and global logic, thus affecting the accuracy and reliability of reasoning.

[0004] To address the aforementioned challenges, there is an urgent need to develop an attention-aware path reasoning method, focusing on breakthroughs in key technologies such as query-guided path discovery and dynamic fusion of multi-source information. This method should aim to achieve two core objectives: first, by modeling edge weights through graph attention networks and combining them with the semantic similarity of query entities, designing a biased path sampling strategy to focus on highly relevant paths; second, by introducing a gating fusion mechanism to adaptively integrate local entity features with global path context, forming a more discriminative comprehensive representation. These advancements are expected to drive the evolution of knowledge graphs from static relational networks to interpretable and reasonable intelligent decision-making systems, providing a crucial technological foundation for knowledge reasoning in complex domains. Summary of the Invention

[0005] This invention addresses the problems of blind path sampling and static information fusion in existing knowledge graph reasoning technologies by proposing an attention-aware path reasoning method for knowledge graphs. This method combines graph attention networks with path reasoning technology and is particularly suitable for fusing local neighborhood information of entities with global path semantics. Through goal-guided path sampling and gated attention fusion mechanism collaborative modeling, it improves the performance of associative reasoning for complex knowledge.

[0006] The technical solution for achieving the objective of this invention is as follows: Firstly, this invention provides an attention-aware path reasoning method for knowledge graphs, comprising:

[0007] Step 1: Encode entities and their adjacency relationships using a graph attention network to generate entity representations containing local semantics, and simultaneously obtain attention weights representing the strength of associations between entities.

[0008] Step 2: Based on the goal-guided biased random walk path sampling strategy, using the above attention weights as biases, adaptively explore high-quality multi-hop semantic paths related to the target task.

[0009] Step 3: Aggregate information from the sampled semantic paths using a path encoder to obtain a global path representation;

[0010] Step 4: Deeply integrate local entity representations and global path representations, and input them together into the prediction layer for knowledge graph link prediction.

[0011] Furthermore, the specific steps of step 1 are as follows:

[0012] Step 1.1: Input each entity in the diagram. initial feature vector ,in For nodes, The input features of the node;

[0013] Step 1.2: Based on the shared weight matrix W and the attention parameter vector a, calculate the attention coefficient of node i for each neighbor node j. ;

[0014] Step 1.3: Use the attention coefficient The output features of the nodes are obtained by weighted summation of the features of neighboring nodes and application of a nonlinear activation function σ.

[0015] Step 1.4: Employ a multi-head attention mechanism to concatenate or average the outputs of K independent attention heads to generate the final graph attention network embedding for the nodes. ;

[0016] Step 1.5: Output the learned attention weights { }, used for subsequent path sampling strategies.

[0017] Furthermore, the specific steps of step 2 are as follows:

[0018] Step 2.1: Given a head entity h and a target tail entity t, start traversing from the head entity h and record the current entity as h. ;

[0019] Step 2.2: Obtain the current entity set of all neighboring nodes ;

[0020] Step 2.3: For each neighbor node The attention weights between the entity and the current entity are calculated comprehensively. and its semantic similarity with the target entity t. Generate an unnormalized transfer preference score;

[0021] Step 2.4: Normalize the transfer preference scores corresponding to all neighboring nodes to obtain the transfer probability distribution from the current entity to each neighboring node;

[0022] Step 2.5: Based on the transition probability distribution, sample and select the next-hop entity, and update the current entity. ;

[0023] Step 2.6: Repeat steps 2.2 to 2.5 until the path termination condition is met, thereby generating a semantic path starting from h;

[0024] Wherein, the attention weight The semantic similarity is generated by the graph attention encoder. It is calculated based on the node embedding vector output by the graph attention encoder.

[0025] Furthermore, the specific steps of step 3 are as follows:

[0026] Step 3.1: For each sampled path p, obtain the feature embedding of each node on the path generated by the graph attention encoder;

[0027] Step 3.2: Use a recurrent neural network to perform bidirectional sequential encoding on the path to obtain the final hidden state in each encoding direction;

[0028] Step 3.3: Concatenate the final hidden states in the two encoding directions to generate a fixed-length vector representation of the path;

[0029] Step 3.4: For multiple paths sampled from the same head and tail entity pair, generate corresponding path vectors according to steps 3.1 to 3.3 respectively;

[0030] Step 3.5: Calculate the attention weight of each path vector using an attention mechanism. The attention weight is used to characterize the importance of the corresponding path to the semantic relationship of the current entity pair.

[0031] Step 3.6: Based on the calculated attention weights, perform a weighted summation of all path vectors to generate a global path context vector for subsequent relational reasoning.

[0032] Furthermore, step 4 consists of the following specific steps:

[0033] Step 4.1: Based on the gating fusion mechanism, the graph attention network embedding of the head entity is dynamically fused with the global path context vector to generate the final feature representation of the head entity;

[0034] Step 4.2: Input the final feature representation of the head entity, the embedding vector of the relationship to be predicted, and the graph attention network embedding of the tail entity into a scoring function for calculation to obtain the matching score of the triple;

[0035] Step 4.3: Construct a set of negative sample triples and use a marginal ranking loss function to train and optimize the model parameters by maximizing the score difference between positive and negative sample triples.

[0036] In a second aspect, the present invention provides an attention-aware path reasoning system for knowledge graphs, used to implement the method described in the first aspect, the system comprising:

[0037] The first module uses a graph attention network to encode entities and their adjacency relationships, generating entity representations that contain local semantics, and simultaneously acquiring attention weights that represent the strength of associations between entities.

[0038] The second module is based on a goal-guided biased random walk path sampling strategy, which uses the aforementioned attention weights as a bias to adaptively explore high-quality multi-hop semantic paths related to the target task.

[0039] The third module is used to aggregate information from the sampled semantic paths through the path encoder to obtain a global path representation.

[0040] The fourth module is used to deeply integrate local entity representations and global path representations, and input them together into the prediction layer for knowledge graph link prediction.

[0041] Thirdly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described in the first aspect.

[0042] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0043] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0044] Compared with the prior art, the significant advantages of the present invention are: (1) The present invention designs a target-guided dynamic biased path sampling strategy, which uses graph attention weights and target semantic similarity to guide the walking direction, significantly improving the efficiency and quality of related path discovery; (2) The present invention proposes a gated attention fusion mechanism, which realizes the adaptive collaboration between local neighborhood information of entities and global path context, enhancing the model's reasoning ability and interpretability for complex logic. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the attention-aware path reasoning method for knowledge graphs according to the present invention;

[0046] Figure 2 This is a framework diagram of the objective-guided biased random walk strategy of the present invention. Detailed Implementation

[0047] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the method flow of the present invention consists of multiple sequentially executed steps, and the output of the previous step is the input of the next step, together forming a complete technical solution.

[0048] Combination Figure 1 , Figure 2 This invention provides an attention-aware path reasoning method for knowledge graphs, aiming to solve the problem that existing link prediction methods struggle to balance the close local associations and complex multi-hop semantic paths in knowledge graphs, leading to inaccurate knowledge delivery. First, a multi-head graph attention network is used to encode the local neighborhood structure of entities, generating node embeddings and attention weights that imply edge importance. Then, a target-guided biased random walk strategy is designed, utilizing the attention weights and combining them with the semantic similarity of the target entity for path sampling to efficiently obtain highly relevant semantic paths. Next, a recurrent neural network is used to encode the path sequence, and an attention mechanism is used to aggregate multiple paths by importance weighting, generating a global path context vector. Finally, a gating fusion mechanism is introduced to dynamically fuse local entity embeddings and global path information, and a scoring function and marginal loss are used for model training and inference. The method specifically includes the following steps:

[0049] Step 1: Entity Neighborhood Encoding Based on Graph Attention Network. This step aims to encode the local neighborhood structure information of each entity in the knowledge graph, generate entity embeddings that contain graph structure relationships, and calculate the importance weights of edges to guide subsequent path discovery.

[0050] Step 1.1: Entity Feature and Parameter Initialization. This step aims to provide the computational foundation for the graph attention network. The input is the initial set of feature vectors for all entity nodes in the knowledge graph. This step prepares the network parameters, including the randomly initialized shared weight matrix W and the attention mechanism parameter vector a. The output of this step, the initialized parameters, will be directly used for the attention coefficient calculation in step 1.2.

[0051] Step 1.2: Calculate the attention coefficients between nodes. This step aims to quantify the association strength between each entity node in the graph and its neighboring nodes. For the current center node... and any of its neighboring nodes ∈N( The features of both are linearly transformed by the weight matrix W. The input is concatenated and fed into a single-layer feedforward neural network consisting of a parameter vector a and a LeakyReLU activation function. A softmax function is then applied to normalize all neighbors of the current node to generate attention coefficients. The specific calculation formula is as follows:

[0052]

[0053] Where N( ) represents a node The neighbor set, where || denotes the vector concatenation operation. The attention coefficients output in this step. This will be used as a weight for feature aggregation in step 1.3.

[0054] Step 1.3: Aggregate neighbor features and generate the final embedding. This step aims to synthesize the node's output features. First, based on the attention coefficients calculated in Step 1.2, { }, for the central node Features of all neighboring nodes after transformation We perform a weighted summation and pass it through a non-linear activation function σ to obtain the output features of the node under one attention head. The specific calculation formula is as follows:

[0055]

[0056] To stabilize the learning process and increase model capacity, K independent steps 1.2 and 1.3 are executed in parallel, employing a K-head attention mechanism. Finally, the output features calculated from the K heads are concatenated (or averaged) to generate nodes. The final graph attention network embedding And record the set of all head attention coefficients { } serves as an important metric for edge embedding. The entity embeddings output in this step are { } and attention weight set { These will be used together as input for subsequent path sampling strategies.

[0057] Step 2: Target-guided biased random walk path sampling. See the appendix of the instruction manual for the specific implementation framework. Figure 2 This step aims to efficiently discover and collect high-quality, highly relevant semantic paths connecting a given query's head and tail entity pairs within the knowledge graph. Its core lies in utilizing the attention weights learned in Step 1 to guide local directions and combining this with semantic similarity to the target to guide the global direction, overcoming the blindness of traditional random walks.

[0058] Step 2.1: Initialize the walk state and parameters. This step aims to set the initial conditions for a single path sampling. The input consists of a head entity h and a tail entity t to be queried, as well as the global attention weights obtained from Step 1. }, All entity graph attention network embeddings { }, Target guidance intensity hyperparameter λ. This refers to the current entity... Initialize the head entity h and initialize an empty path. The output of this step, i.e. the initialized walk state, will be directly used for neighbor exploration in step 2.2.

[0059] Step 2.2: Obtain the neighbor set of the current entity. This step aims to determine the candidate next steps for path sampling. Based on the topology of the knowledge graph, retrieve the current entity... All outgoing neighbors constitute its neighbor set. The neighbor set output in this step. This will be the direct object of the transfer score calculation in step 2.3.

[0060] Step 2.3: Calculate the transition score for neighboring nodes. This step aims to calculate an unnormalized transition preference score for each candidate node in the neighbor set. For each neighboring node... ,score It consists of two parts: one is the edge representation obtained from step 1. Attention weights based on local importance Secondly, the neighboring node Semantic similarity with target tail entity t in the graph attention network embedding space This similarity is calculated using the cosine similarity function, specifically:

[0061]

[0062] in Representing neighboring nodes Embedded representation after encoding in a graph attention network This represents the embedding representation of the target tail entity t after encoding in the graph attention network.

[0063] The two parts are weighted and summed using the hyperparameter λ, and the calculation formula is as follows:

[0064]

[0065] This step outputs the set of unnormalized scores for all neighbors. This will serve as the basis for the probability transformation in step 2.4.

[0066] Step 2.4: Normalize and sample the next-hop entity. This step aims to determine the specific transition behavior based on the score, completing one step of path expansion. First, normalize and sample the unnormalized scores of all neighboring nodes calculated in Step 2.3. The softmax function is applied for normalization to obtain a probability distribution. Then, based on this probability distribution, the next entity is selected as the next hop of the path through random sampling. The selected next entity output in this step will be used to update the current entity and added to the path being constructed.

[0067] Step 2.5: Update the state and iterate. This step aims to iteratively execute the walkthrough to generate a complete path. Update the next entity sampled in Step 2.4 with the new current entity. Then, add the entity to the path sequence. Repeat steps 2.2 to 2.5 until the current entity equals the target tail entity t, or the path length reaches a preset threshold, at which point a complete semantic path is generated. For the same entity pair (h, t), repeat step 2 multiple times to collect multiple paths and form a path set.

[0068] Step 3: Path Encoding and Weighted Aggregation. This step aims to encode the serialized paths sampled in Step 2, consisting of entities and relationships, into fixed-dimensional vector representations, and to use an attention mechanism to distinguish the importance of different paths, aggregating them to generate a comprehensive global path semantic representation.

[0069] Step 3.1: Sequence Encoding of a Single Path. This step aims to transform the sequence information of a single path into a comprehensive vector representation. The input is a single path. (in , , The vector of entity nodes at the start of the path. Let r be the vector of the endpoint node of the path, and r be the vector of the relationship between any two entities in the path. The graph attention network embeddings of all nodes on the path generated in step 1 are also included. The sequence is processed using a bidirectional recurrent neural network (specifically, a bidirectional gated recurrent unit, BiGRU). The embeddings of each node along the path are sequentially input into the network, and forward and backward encodings are performed to obtain the bidirectional hidden state at each position in the sequence. Finally, the last hidden state from the forward propagation is concatenated with the starting hidden state from the backward propagation to obtain a fixed-length vector representation p representing the entire path. The path vector p output in this step will serve as the basic unit for path importance evaluation in step 3.2.

[0070] Step 3.2: Multi-path attention weight calculation. This step aims to evaluate the relative importance of multiple paths corresponding to the same entity pair (h, t). The input is the set of vector representations of the M paths generated in Step 3.1 { }, and the learnable parameter matrix , bias vector b and query vector q.

[0071] For the vector of the m-th path First, through a feedforward network (computing) Map it to a scalar fraction This score characterizes the initial importance of the path. Subsequently, the scores for all M paths are { The attention weights are obtained by applying the softmax function for normalization. The calculation formula is:

[0072]

[0073] The attention weights output in this step { This will serve as the basis for weighted aggregation of path vectors in step 3.3.

[0074] Step 3.3: Generate global path context vectors. This step aims to fuse information from multiple paths to form a unified, enhanced path semantic representation. This is based on the attention weights for each path calculated in Step 3.2. For the corresponding path vector Perform a weighted summation to calculate the global path context vector. The calculation formula is:

[0075]

[0076] This vector generation process allows the model to dynamically focus on paths more relevant to the current inference and suppress the influence of noisy paths. The output of this step is a global path context vector. This will serve as a key input for the subsequent information fusion module.

[0077] Step 4: Information Fusion and Model Training Inference. This step aims to effectively fuse the entity's local neighborhood information with the global semantic information extracted from multi-hop paths to form a more discriminative comprehensive representation, and then use this representation for knowledge graph completion prediction and model training.

[0078] Step 4.1: Multi-source information fusion based on gating mechanism. This step aims to adaptively fuse the local structural information and global path information of the head entity. The input is the graph attention network embedding of the head entity h. Compared with the global path context vector generated in step 3 .

[0079] First, the two vectors are concatenated and input into a fully connected layer, and a gated vector g is generated by the Sigmoid activation function σ, the calculation formula of which is as follows:

[0080]

[0081] in and For learnable parameters, [;] denotes vector concatenation. Each element of the gating vector g has a value between 0 and 1, used to control the proportion of information from both sides. Finally, the two types of information are weighted and combined using the gating vector to generate the final fused head entity representation. The calculation formula is:

[0082]

[0083] Where ⊙ denotes element-wise multiplication. This step dynamically adjusts the contributions of different information sources, and the output... This will be used for subsequent triple modeling.

[0084] Step 4.2: Triple Score Calculation. This step aims to evaluate the reasonableness of a given triple (h, r, t) consisting of a head entity, relation, and tail entity. The input is the fused head entity representation obtained in Step 4.1. The embedding vector of the relation r to be predicted, and the graph attention network embedding of the tail entity t. A scoring function φ is used to calculate the matching score for the triplet. Specifically, the DistMult function is chosen as the scoring function, and the score is calculated using a trilinear dot product, as shown in the formula:

[0085]

[0086] This represents the score of the triple, where h represents the head entity, r represents the relation, t represents the tail entity, and i represents each dimension of the traversal vector. The scalar score output in this step indicates that the triple is more likely to be true.

[0087] Step 4.3: Negative Sampling and Model Training Optimization. This step aims to train the model parameters by optimizing the loss function. In each training iteration, the model parameters are extracted from the training set... Select a positive triplet (h, r, t). By replacing its head or tail entity, construct a set of negative triplets. The goal of model training is to ensure that the score of a positive sample is higher than the score of its corresponding negative sample by at least a marginal value γ. This is optimized using a marginal ranking loss function, calculated as follows:

[0088]

[0089] By minimizing the loss function through backpropagation, all learnable parameters of the model are updated, including graph attention network parameters, path encoder parameters, attention aggregation parameters, gating fusion parameters, and relation embedding vectors, ultimately resulting in a model capable of accurately reasoning about missing links in a knowledge graph.

[0090] The above embodiments are descriptions of specific implementations of the present invention, and not limitations thereof. Those skilled in the art can make various modifications and changes without departing from the spirit and scope of the present invention to obtain corresponding equivalent technical solutions. Therefore, all equivalent technical solutions should be included in the protection scope of the present invention.

Claims

1. A knowledge graph attention-aware path reasoning method, characterized in that, include: Step 1: Encode entities and their adjacency relationships using a graph attention network to generate entity representations containing local semantics, and simultaneously obtain attention weights representing the strength of associations between entities. Step 2: Based on the goal-guided biased random walk path sampling strategy, using the above attention weights as biases, adaptively explore high-quality multi-hop semantic paths related to the target task. Step 3: Aggregate information from the sampled semantic paths using a path encoder to obtain a global path representation; Step 4: Deeply integrate local entity representations and global path representations, and input them together into the prediction layer for knowledge graph link prediction.

2. The method according to claim 1, characterized in that, Step 1: The specific steps are as follows: Step 1.1: Input each entity in the diagram. initial feature vector ,in For nodes, The input features of the node; Step 1.2: Based on the shared weight matrix W and the attention parameter vector a, calculate the attention coefficient of node i for each neighbor node j. ; Step 1.3: Use the attention coefficient The output features of the nodes are obtained by weighted summation of the features of neighboring nodes and application of a nonlinear activation function σ. Step 1.4: Employ a multi-head attention mechanism to concatenate or average the outputs of K independent attention heads to generate the final graph attention network embedding for the nodes. ; Step 1.5: Output the learned attention weights { }, used for subsequent path sampling strategies.

3. The method according to claim 1, characterized in that, Step 2 is detailed below: Step 2.1: Given a head entity h and a target tail entity t, start traversing from the head entity h and record the current entity as h. ; Step 2.2: Obtain the current entity set of all neighboring nodes ; Step 2.3: For each neighbor node The attention weights between the entity and the current entity are calculated comprehensively. and its semantic similarity with the target entity t. Generate an unnormalized transfer preference score; Step 2.4: Normalize the transfer preference scores corresponding to all neighboring nodes to obtain the transfer probability distribution from the current entity to each neighboring node; Step 2.5: Based on the transition probability distribution, sample and select the next-hop entity, and update the current entity. ; Step 2.6: Repeat steps 2.2 to 2.5 until the path termination condition is met, thereby generating a semantic path starting from h; Wherein, the attention weight The semantic similarity is generated by the graph attention encoder. It is calculated based on the node embedding vector output by the graph attention encoder.

4. The method according to claim 1, characterized in that, Step 3 is detailed below: Step 3.1: For each sampled path p, obtain the feature embedding of each node on the path generated by the graph attention encoder; Step 3.2: Use a recurrent neural network to perform bidirectional sequential encoding on the path to obtain the final hidden state in each encoding direction; Step 3.3: Concatenate the final hidden states in the two encoding directions to generate a fixed-length vector representation of the path; Step 3.4: For multiple paths sampled from the same head and tail entity pair, generate corresponding path vectors according to steps 3.1 to 3.3 respectively; Step 3.5: Calculate the attention weight of each path vector using an attention mechanism. The attention weight is used to characterize the importance of the corresponding path to the semantic relationship of the current entity pair. Step 3.6: Based on the calculated attention weights, perform a weighted summation of all path vectors to generate a global path context vector.

5. The method according to claim 1, characterized in that, Step 4 is detailed below: Step 4.1: Based on the gating fusion mechanism, the graph attention network embedding of the head entity is dynamically fused with the global path context vector to generate the final feature representation of the head entity; Step 4.2: Input the final feature representation of the head entity, the embedding vector of the relationship to be predicted, and the graph attention network embedding of the tail entity into a scoring function for calculation to obtain the matching score of the triple; Step 4.3: Construct a set of negative sample triples and use a marginal ranking loss function to train and optimize the model parameters by maximizing the score difference between positive and negative sample triples.

6. A knowledge graph-based attention-aware path reasoning system, characterized in that, The system for implementing the method according to any one of claims 1 to 5 comprises: The first module uses a graph attention network to encode entities and their adjacency relationships, generating entity representations that contain local semantics, and simultaneously acquiring attention weights that represent the strength of associations between entities. The second module is based on a goal-guided biased random walk path sampling strategy, which uses the aforementioned attention weights as a bias to adaptively explore high-quality multi-hop semantic paths related to the target task. The third module is used to aggregate information from the sampled semantic paths through the path encoder to obtain a global path representation. The fourth module is used to deeply integrate local entity representations and global path representations, and input them together into the prediction layer for knowledge graph link prediction.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of any of the methods described in claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-5.

9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-5.

Images