Multi-hop knowledge graph reasoning method based on semantic and topology enhancement

Abstract

The invention relates to the field of deep reinforcement learning technology and knowledge graph reasoning, discloses a multi-hop knowledge graph reasoning method based on semantic and topology enhancement, and aims to improve reasonability and interpretability of a reasoning path when a head entity and a relation are given to reasone a tail entity. The system comprises a path acquisition module and a reward calculation module. Firstly, a reinforcement learning algorithm is used to control an agent to obtain a reasoning path; secondly, in order to improve the reasonability of the reasoning path, the semantic consistency of the path and the multi-hop intermediary centrality of the entity are constrained; specifically, by calculating the local similarity and global tightness of a relationship and an entity, the consistency of the relationship and the entity is measured respectively, so that the semantic consistency of the path is measured; by calculating multi-hop intermediary centrality scores of entities in a reasoning path, key entities are identified, and a more reasonable reasoning path is selected. And finally, fusing the two scores to obtain a reasonable reward function, and guiding the intelligent agent to select an accurate and reasonable reasoning path in combination with an accurate reward calculated according to a reasoning result.

Description

Technical Field

[0001] The present invention relates to the field of deep reinforcement learning technology and knowledge graph reasoning, and specifically to a multi-hop knowledge graph reasoning method based on semantic and topological enhancement. Background Art

[0002] Knowledge Graph (KG) is a technology that represents fact triples in a structured form and has been widely used in downstream tasks such as recommendation systems and question-answering systems. However, most knowledge graphs are incomplete and lack many facts, which limits their effectiveness in downstream tasks. To address this problem, knowledge graph reasoning (KGR) technology has received widespread attention. Initial embedding methods capture implicit relationships and predict missing facts by learning representations of fact triples in high-dimensional vector spaces. Although these methods perform well in predictive performance, they are insufficient in interpretability. Rule reasoning methods reason by mining logical rules in knowledge graphs. Although they have advantages in interpretability, they have limited expressive power and high computational complexity. Multi-hop reasoning methods regard the reasoning process as query answering, and can search for paths in the existing knowledge graph to infer the missing target entities and provide clear answers for step-by-step reasoning. However, multi-hop reasoning methods also have the problem of false paths, that is, the model often ignores the rationality of the path during the learning process, resulting in a large number of unreasonable paths being inferred. Summary of the invention

[0003] To solve the above problems, this paper designs a multi-hop knowledge graph reasoning method based on semantic and topological enhancement. By introducing the semantic consistency constraint module and the multi-hop betweenness centrality constraint module, the relationships and entities in the path are ensured to maintain high semantic consistency, and the rationality and accuracy of the model in the reasoning path selection are enhanced. The multi-hop betweenness centrality constraint module optimizes the path selection by calculating the key entities in the reasoning path. In addition, these constraints are integrated into a reasonable reward function to guide the agent to select the optimal reasoning path in the dynamically changing knowledge graph.

[0004] To achieve the above object, the present invention provides the following technical solution: a multi-hop knowledge graph reasoning method based on semantic and topological enhancement, wherein the interference resource allocation method comprises the following specific steps:

[0005] Step S10: Use the reinforcement learning algorithm to obtain the reasoning path. By embedding the entities and relationships in the knowledge graph and parameterizing them in combination with the state information, query relationship information, and historical search path, the agent selects actions at each step to construct the reasoning path, and finally obtains a reasoning path from the source entity to the target entity.

[0006] Step S20: By calculating the local relationship similarity LRS and the global relationship closeness GRC for the relationships in the reasoning path, and calculating the local entity similarity LES and the global entity closeness GEC for the entities, the semantic consistency scores RC and EC of the relationships and entities are obtained, and then the semantic consistency score SC of the entire reasoning path is obtained comprehensively, ensuring that the relationships and entities in the path maintain high semantic consistency;

[0007] Step S30: by constructing an adjacency matrix for the entities in the knowledge graph and calculating the number of multi-hop paths, further calculating the betweenness centrality BC of the entities, identifying the irreplaceable key entities in the reasoning path, and thus introducing the multi-hop betweenness centrality constraint;

[0008] Step S40: Integrate the semantic consistency constraint and the multi-hop betweenness centrality constraint into the reward function of reinforcement learning to optimize the selection of the reasoning path.

[0009] Preferably, the specific steps of step S10 are as follows:

[0010] Step S101: The state of the reinforcement learning agent, the state s at step t t =(e t , e s , r q )∈S, by the entity e at the tth step t , source entity e s and query relation r q Composition, S represents the set of all possible states:

[0011] Step S102: Reinforce the actions of the learning agent, the action set A of step t t ={(r′ t ,e′ t )|(e t , r′ t ,e′ t )∈G},A t ∈A, is the neighbor set of the current entity, which is composed of the outward edge of the current entity and the corresponding entity, where r′ t Indicates the relationship selected by the agent at step t, e′ t Indicates the next entity selected by the agent at step t, e t represents the entity that the agent is currently in at step t, and G represents the entire knowledge graph. We represent the reasoning process as:

[0012]

[0013] where e s is the source entity, e T Indicates the entity at the last time step, r i(i=1…T) represents the relationship corresponding to the i-th time step;

[0014] Step S103: Accuracy reward function, if the agent finally stays on the accurate entity, that is, e T =e o (e o represents the target entity), then the accuracy reward R s (S T )=R b (S T )=1, otherwise the pre-trained embedding model ConvE is used to calculate the probability f(e s , r q , e T ) as the accuracy reward, the specific form of the accuracy reward function is as follows:

[0015] R s (s T )=R b (s T )+(1-R b (s T ))f(e s ,r q ,e T )

[0016] Where R b (S T ) indicates state S T The corresponding accuracy reward is (e s ,r q ,e T ) represents the triplet obtained by final reasoning;

[0017] Step S104: Strategy function. The agent’s search strategy is parameterized by state information, query relationship information, and historical search path. Specifically, each entity and relationship in the knowledge graph is represented by embedding, and GRU is used to encode the historical path. The path encoding is in the following form:

[0018] h0=GRU(0,a0)

[0019] h t =GRU(h t-1 ,a t-1 ), t>0

[0020] Where GRU(*) represents the gated recurrent unit, h0 represents the initial path encoding state, a0 is the default initial action, and h t represents the path encoding state at time t, a t-1 represents the action chosen by the agent at time t-1;

[0021] The parameterized policy network can be expressed as:

[0022] π θ (a t+1 |s t )=softmax(A t W2ReLU(W1[h t ;e t ; r q ]))

[0023] ReLU: represents a nonlinear activation function, that is, when the input is negative, the output is zero, and when the input is positive, the output is the input value;

[0024] Softmax: represents the normalization function; W1 represents the weight matrix, which is used to encode the path h t Entity e t and query relation r q Combined into a vector; W1 represents the weight matrix, which is used to transform the output of W1, a t+1 represents the action selected by the agent at time t+1; A t represents the action space of the agent at time t, π θ (a t+1 |s t ) is in state s t Play A t The probability distribution of all actions in .

[0025] Preferably, the specific steps of step S20 are as follows:

[0026] Step S201: Calculate the local relation similarity LRS, and use the normalized cosine similarity to calculate the local relation similarity LRS of each relation r in the reasoning path. t Relationship with query q similarity to ensure that the relationships in the path maintain high semantic consistency;

[0027]

[0028] in r t , r q The vector representation of , T is the total length of the relationship or entity sequence;

[0029] Step S202: Calculate the global relationship closeness GRC by calculating each selected relationship r b The degree to which the distance between the target entity and the source entity is reduced measures the global semantic closeness of the relationship in the path;

[0030]

[0031] Cnt -Δ (r b ) indicates that the relationship r is selected in the path b The number of distances that decrease after that, Cnt +Δ (r b ) indicates that the relationship r is selected in the path b The number of distances that increase afterward;

[0032] Step S203: Combining the local relationship similarity with the global relationship closeness to obtain the semantic consistency score RC of the relationship:

[0033]

[0034] Step S204: Calculate the local entity similarity LES, and calculate the t With the target entity e o To ensure that entities in the path maintain high semantic consistency:

[0035]

[0036] in Represents entity e t With the target entity e o The vector representation of ;

[0037] Step S205: Calculate the global entity closeness GEC, by calculating the degree to which each selected entity reduces the distance between the target entity and the source entity, to measure the global semantic closeness of the entities in the path:

[0038]

[0039] Cnt -Δ (e b ) indicates the selection relationship e in the path b The number of distances that decrease after that, Cnt +Δ (e b ) indicates the selection relationship e in the path b The number of distances that increase afterward;

[0040] Step S206: Combining the local entity similarity and the global entity closeness to obtain the entity's semantic consistency score EC:

[0041]

[0042] Step S207: The semantic consistency scores of the relationship and entity are integrated to obtain the semantic consistency score SC of the entire reasoning path:

[0043] SC=RC×EC

[0044] Preferably, the specific steps of step S30 are as follows:

[0045] Step S301: construct an adjacency matrix, represent the entities and relationships in the knowledge graph as an adjacency matrix, and if there is a relationship between entities, the value of the corresponding position in the matrix is ​​increased by one;

[0046] Step S302: Calculate the number of multi-hop paths, use the adjacency matrix to calculate the number of multi-hop relationship paths from each entity in the reasoning path to the target entity, and identify key entities in the path;

[0047] Step S303: Calculate the betweenness centrality BC, by counting the number of multi-hop paths, calculate the betweenness centrality of each entity in the reasoning path, and identify the key entities that cannot be replaced in the path reasoning process:

[0048]

[0049] in Represents the intermediate entity e t To the entity e at the last time step T The number of paths of length i, Represents all entities to the target entity e T The number of paths of length i, BC(e t ) measures the intermediate entity e t Importance in topology.

[0050] Preferably, the specific steps of step S40 are as follows:

[0051] Step S401: Design accuracy reward function R S , which combines a binary reward function and a soft reward function; the binary reward function gives the agent a positive reward when the reasoning path is correct, otherwise it gives a negative reward; the soft reward function uses a pre-trained embedding model to calculate the soft reward of the reasoning path, providing more refined feedback to the agent:

[0052] R s (s T )=R b (s T )+(1-R b (s T ))×f(e s , r q , e T )

[0053] Where (1-R b (s T ))×f(e s , r q , e T ) is the soft reward function, Rb (s T ) is a binary reward function;

[0054] Step S402: Design rationality reward function R r , combining the semantic consistency constraint and the multi-hop betweenness centrality constraint, the rationality reward of the reasoning path is calculated:

[0055] R r (r t ,e t )=αSC(r t ,e t )×(1-α)BC(e t )

[0056] Among them, α is the weight parameter, e t To calculate the entities in the reasoning path, SC(r t ,e t ) is through the comprehensive relationship r t and entity e t The semantic consistency score is calculated to measure the consistency of relations and entities in the path, BC(e t ) is calculated by calculating the entity e in the reasoning path t The betweenness centrality is used to identify key entities in the path;

[0057] Step S403: The final reward function R is obtained by integrating the accuracy reward and the rationality reward. The binary reward, the soft reward and the rationality reward are integrated into the final reward function. By adjusting the relative importance weights of each part, the selection of the reasoning path is optimized to guide the agent to select the optimal path.

[0058] R((r t , e t ), s T )=R s (s T )×(1+R r (r t ,e t ))

[0059] Step S404: The policy gradient optimization algorithm updates the policy network and trains the policy network by maximizing the expected return of all queries in the knowledge graph through gradient ascent, where the cumulative expected return is:

[0060]

[0061] where π θ is the agent’s policy network, a1,…a T The agent executes the query triple (e s ,r,eo ) process, The meaning is that all triples (e s ,r,e o )’s expected return, The meaning is that in strategy π θ Under the guidance of action sequence a1,...a T expected value.

[0062] The REINFORCE algorithm is used to solve the optimization problem and traverse all query triples in the knowledge graph (e s ,r,e o ) Update the parameters of the policy network by the following formula:

[0063]

[0064] Represents the policy network π θ The derivative of the parameter θ, R t represents the reward obtained by the agent at time t, logπ θ (a t |s t ) represents the policy network π θ In status t Take action a t The logarithmic probability of . is the policy gradient approximated using REINFORCE, and θ′ is the updated parameter.

[0065] Because the present invention adopts the above technical solution, it has the following beneficial effects:

[0066] 1. This invention solves the problem of false paths in knowledge graph multi-hop reasoning technology by means of semantic and topological enhancement.

[0067] It not only achieves higher reasoning accuracy but also improves the explainability of the reasoning path.

[0068] 2. The present invention constrains the interpretability of the path from two dimensions: semantic space and topological space, which solves the problem of insufficient comprehensiveness of only considering features in a single space and improves the generalization ability of the method.

[0069] 3. The reward function designed by the present invention integrates the accuracy and rationality reward functions, solving the problem that the traditional multi-hop reasoning method based on reinforcement learning only has the accuracy reward function but does not consider the rationality of the path. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 The figure is a flow chart of the steps of the method of the present invention. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] See also Figure 1 , the present invention provides a technical solution:

[0073] The present invention provides the following technical solution: an adaptive interference resource allocation method based on a perception enhancement network, wherein the specific steps of the interference resource allocation method are as follows:

[0074] Step 1: Use the reinforcement learning algorithm to obtain the reasoning path. By embedding the entities and relationships in the knowledge graph and parameterizing them with state information, query relationship information, and historical search paths, the agent selects actions at each step to build a reasoning path, and finally obtains a reasoning path from the source entity to the target entity.

[0075] Step 1-1: The state of the reinforcement learning agent, the state s at step t t =(e t , e s , r q )∈S, by the entity e at the tth step t , source entity e s and query relation r q Composition, S represents the set of all possible states;

[0076] Step 1-2: Reinforcement learning agent’s actions, action set A at step t t ={(r′ t , e′ t )|(e t , r′ t , e′ t )∈G},A t ∈A, is the neighbor set of the current entity, which is composed of the outward edge of the current entity and the corresponding entity, where r′ t Indicates the relationship selected by the agent at step t, e′ t Indicates the next entity selected by the agent at step t, e t represents the entity that the agent is currently in at step t, and G represents the entire knowledge graph. We represent the reasoning process as:

[0077]

[0078] where e s is the source entity, e T Indicates the entity at the last time step, r i (i=1…T) represents the relationship corresponding to the i-th time step;

[0079] Step 1-3: Accuracy reward function, if the agent finally stays on the accurate entity, that is, e T =e o (e o represents the target entity), then the accuracy reward R s (S T )=R b (S T )=1, otherwise the pre-trained embedding model ConvE is used to calculate the probability f(e s ,r q ,e T ) as the accuracy reward, the specific form of the accuracy reward function is as follows:

[0080] R s (s T )=R b (s T )+(1-R b (s T ))f(e s ,r q ,e T )

[0081] Where R b (S T ) indicates state S T The corresponding accuracy reward is (e s ,r q ,e T ) represents the triplet obtained by final reasoning;

[0082] Step 1-4: Strategy function. The agent’s search strategy is parameterized by state information, query relationship information, and historical search path. Specifically, embedding is used to represent each entity and relationship in the knowledge graph, and GRU is used to encode the historical path. The path encoding is in the following form:

[0083] h0=GRU(0,a0)

[0084] h t =GRU(h t-1 , a t-1 ), t>0

[0085] Where GRU(*) represents the gated recurrent unit, h0 represents the initial path encoding state, a0 is the default initial action, and ht represents the path encoding state at time t, a t-1 represents the action chosen by the agent at time t-1;

[0086] The parameterized policy network can be expressed as:

[0087] π θ (a t+1 |s t )=softmax(A t W2ReLU(W1[h t ;e t ; r q ]))

[0088] ReLU: represents a nonlinear activation function, that is, when the input is negative, the output is zero, and when the input is positive, the output is the input value;

[0089] Softmax: represents the normalization function; W1 represents the weight matrix, which is used to encode the path h t Entity e t and query relation r q Combined into a vector; W1 represents the weight matrix, which is used to transform the output of W1, a t+1 Represents the action chosen by the agent at time t+1: A t represents the action space of the agent at time t, π θ (a t+1 |s t ) is in state s t Play A t The probability distribution of all actions in .

[0090] Step 2: By calculating the local relationship similarity LRS and global relationship closeness GRC for the relationships in the reasoning path, and calculating the local entity similarity LES and global entity closeness GEC for the entities, the semantic consistency scores RC and EC of the relationships and entities are obtained, and then the semantic consistency score SC of the entire reasoning path is obtained comprehensively to ensure that the relationships and entities in the path maintain high semantic consistency:

[0091] Step 2-1: Calculate the local relation similarity LRS, using the normalized cosine similarity to calculate each relation r in the reasoning path t Relationship with query q similarity to ensure that the relationships in the path maintain high semantic consistency;

[0092]

[0093] in r t ,r qThe vector representation of , T is the total length of the relationship or entity sequence;

[0094] Step 2-2: Calculate the global relationship closeness GRC by calculating each selected relationship r b The degree to which the distance between the target entity and the source entity is reduced measures the global semantic closeness of the relationship in the path;

[0095]

[0096] Cnt -Δ (r b ) indicates that the relationship r is selected in the path b The number of distances that decrease after that, Cnt +Δ (r b ) indicates that the relationship r is selected in the path b The number of distances that increase later:

[0097] Step 2-3: Combine the local relationship similarity and the global relationship closeness to obtain the semantic consistency score RC of the relationship:

[0098]

[0099] Step 2-4: Calculate the local entity similarity LES, and calculate each entity e in the reasoning path t With the target entity e o To ensure that entities in the path maintain high semantic consistency:

[0100]

[0101] in Represents entity e t With the target entity e o The vector representation of ;

[0102] Step 2-5: Calculate the global entity closeness GEC, which measures the global semantic closeness of the entities in the path by calculating the extent to which each selected entity reduces the distance between the target entity and the source entity:

[0103]

[0104] Cnt -Δ (e b ) indicates the selection relationship e in the path b The number of distances that decrease after that, Cnt +Δ (e b ) indicates the selection relationship e in the path b The number of distances that increase afterward;

[0105] Step 2-6: Combine local entity similarity and global entity closeness to obtain the entity's semantic consistency score EC:

[0106]

[0107] Step 2-7: Combine the semantic consistency scores of the relationship and entity to obtain the semantic consistency score SC of the entire reasoning path:

[0108] SC=RC×EC

[0109] Step 3: By constructing an adjacency matrix for the entities in the knowledge graph and calculating the number of multi-hop paths, the betweenness centrality BC of the entities is further calculated to identify the irreplaceable key entities in the reasoning path, thereby introducing multi-hop betweenness centrality constraints;

[0110] Step 3-1: Construct an adjacency matrix, and represent the entities and relationships in the knowledge graph as an adjacency matrix. If there is a relationship between entities, the value of the corresponding position in the matrix is ​​increased by one;

[0111] Step 3-2: Calculate the number of multi-hop paths, use the adjacency matrix to calculate the number of multi-hop relationship paths from each entity in the reasoning path to the target entity, and identify the key entities in the path;

[0112] Step 3-3: Calculate the betweenness centrality BC. By counting the number of multi-hop paths, calculate the betweenness centrality of each entity in the reasoning path and identify the key entities that are irreplaceable in the path reasoning process:

[0113]

[0114] in Represents the intermediate entity e t To the entity e at the last time step T The number of paths of length i, Represents all entities to the target entity e T The number of paths of length i, BC(e t ) measures the intermediate entity e t Importance in topology.

[0115] Step 4: The semantic consistency constraint and multi-hop betweenness centrality constraint are integrated into the reward function of reinforcement learning to optimize the selection of reasoning paths.

[0116] Step 4-1: Design the accuracy reward function R S , which combines a binary reward function and a soft reward function; the binary reward function gives the agent a positive reward when the reasoning path is correct, otherwise it gives a negative reward; the soft reward function uses a pre-trained embedding model to calculate the soft reward of the reasoning path, providing more refined feedback to the agent:

[0117] R s (s T )=R b (s T )+(1-R b (s T ))×f(e s ,r q ,e T )

[0118] Where (1-R b (s T ))×f(e s ,r q ,e T ) is the soft reward function, R b (s T ) is a binary reward function;

[0119] Step 4-2: Design a reasonable reward function R r , combining the semantic consistency constraint and the multi-hop betweenness centrality constraint, the rationality reward of the reasoning path is calculated:

[0120] R r (r t ,e t )=αSC(r t ,e t )×(1-α)BC(e t )

[0121] Among them, α is the weight parameter, e t To calculate the entities in the reasoning path, SC(r t ,e t ) is through the comprehensive relationship r t and entity e t The semantic consistency score is calculated to measure the consistency of relations and entities in the path, BC(e t ) is calculated by calculating the entity e in the reasoning path t The betweenness centrality is used to identify key entities in the path:

[0122] Step 4-3: The final reward function R is obtained by integrating the accuracy reward and the rationality reward. The binary reward, soft reward and rationality reward are integrated into the final reward function. By adjusting the relative importance weights of each part, the selection of the reasoning path is optimized to guide the agent to select the optimal path.

[0123] R((r t ,e t ), s T )=R s (s T)×(1+R r (r t ,e t ))

[0124] Step 4-4: The policy gradient optimization algorithm updates the policy network and trains the policy network by maximizing the expected return of all queries in the knowledge graph through gradient ascent, where the cumulative expected return is:

[0125]

[0126] where π θ is the agent’s policy network, a1,...a T The agent executes the query triple (e s ,r,e o ) process, The meaning is that all triples (e s ,r,e o )’s expected return, The meaning is that in strategy π θ Under the guidance of action sequence a1,...a T expected value.

[0127] The REINFORCE algorithm is used to solve the optimization problem and traverse all query triples in the knowledge graph (e s ,r,e o ) Update the parameters of the policy network by the following formula:

[0128]

[0129] Represents the policy network π θ The derivative of the parameter θ, R t represents the reward obtained by the agent at time t, logπ θ (a t |s t ) represents the policy network π θ In status t Take action a t The logarithmic probability of . is the policy gradient approximated using REINFORCE, and θ′ is the updated parameter.

[0130] A multi-hop knowledge graph reasoning system based on semantic and topological enhancement, including a semantic consistency constraint module, a multi-hop betweenness centrality constraint module and a rationality reward function. Specifically:

[0131] First, the semantic consistency score of the reasoning path is calculated, which is composed of the semantic consistency score of the relationship and the semantic consistency score of the entity. For the relationship semantic consistency score, we obtain it by calculating the local similarity and global closeness; then it is input into the multi-hop betweenness centrality constraint module to calculate the number of multi-hop relationship paths of each entity in the reasoning path and identify key entities; finally, the semantic consistency constraint and the multi-hop betweenness centrality constraint are integrated into the rationality reward function to guide the agent to choose a reasonable reasoning path.

[0132] The method and system of the present invention take the reasonable reasoning process of knowledge graph as the research object, identify the key entities in the reasoning path by enhancing the ability to perceive the semantic consistency of relationships and entities in the path, more accurately match the optimal reasoning path, and improve the rationality and interpretability of the reasoning; and construct an optimization reward function that integrates semantic and topological information, and achieve the optimal balance in reasoning path selection by adjusting the relative importance weights.

[0133] The present invention has the following advantages:

[0134] 1. Reinforcement learning algorithm: Reinforcement learning algorithm simulates the process of interaction between the agent and the environment, enabling the agent to select the best action based on the current state to obtain the maximum cumulative reward. In knowledge graph reasoning, the agent selects the reasoning path based on state information, query relationship information, and historical search paths. This solves the limitation of traditional methods that path selection relies on predefined rules or heuristic algorithms, realizes a more dynamic and adaptive path search, and improves the accuracy of path selection.

[0135] 2. Semantic consistency score calculation: The semantic consistency score calculation ensures the consistency of elements in the path by analyzing the semantic similarity of relationships and entities in the path. This method solves the problem of unreasonable relationship and entity combination that may appear in traditional paths, and enhances the rationality and interpretability of the path. Specifically, by calculating the local relationship similarity and global relationship closeness, as well as the local entity similarity and global entity closeness, the semantic consistency of the path is comprehensively evaluated, so as to select a more semantically reasonable path.

[0136] 3. Multi-hop betweenness centrality calculation: Multi-hop betweenness centrality calculation identifies key entities that are irreplaceable in the path reasoning process. This method solves the shortcomings of traditional methods that fail to identify and utilize key entities and optimizes path selection. By constructing an adjacency matrix and calculating the number of multi-hop paths, key entities that play a bridging role in the path can be identified, thereby selecting a more reasonable path.

[0137] 4. Reward function design: The reward function design integrates accuracy and rationality rewards, solving the problem that traditional methods only consider path accuracy but ignore path rationality. By designing accuracy reward function and rationality reward function and integrating them into the final reward function, the accuracy and rationality of path selection are comprehensively considered, thus guiding the agent to select the optimal reasoning path.

[0138] 5. Policy gradient optimization: The policy gradient optimization algorithm trains the agent’s policy network by maximizing the expected return of all queries in the knowledge graph. This method solves the problem of the inability to dynamically adjust the path selection strategy in traditional methods, and achieves continuous optimization of the policy network, thereby improving the accuracy and rationality of path selection.

[0139] In summary, the present invention solves the irrationality and interpretability problems of path selection in multi-hop reasoning of knowledge graphs through multiple means such as reinforcement learning algorithm, semantic consistency score calculation, multi-hop betweenness centrality calculation, reward function design and policy gradient optimization, while achieving higher reasoning accuracy and improving the interpretability of the reasoning path.

[0140] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-hop knowledge graph reasoning method based on semantic and topological enhancement, It is characterized in that The following steps are involved: Step S10: Use the reinforcement learning algorithm to obtain the reasoning path. By embedding the entities and relationships in the knowledge graph and parameterizing them in combination with the state information, query relationship information, and historical search path, the agent selects actions at each step to construct the reasoning path, and finally obtains a reasoning path from the source entity to the target entity. Step S20: By calculating the local relationship similarity LRS and the global relationship closeness GRC for the relationships in the reasoning path, and calculating the local entity similarity LES and the global entity closeness GEC for the entities, the semantic consistency scores RC and EC of the relationships and entities are obtained, and then the semantic consistency score SC of the entire reasoning path is obtained comprehensively, ensuring that the relationships and entities in the path maintain high semantic consistency; Step S30: by constructing an adjacency matrix for the entities in the knowledge graph and calculating the number of multi-hop paths, further calculating the betweenness centrality BC of the entities, identifying the irreplaceable key entities in the reasoning path, and thus introducing the multi-hop betweenness centrality constraint; Step S40: Integrate the semantic consistency constraint and the multi-hop betweenness centrality constraint into the reward function of reinforcement learning to optimize the selection of the reasoning path.

2. According to claim 1, the multi-hop knowledge graph reasoning method based on semantic and topological enhancement, Features: The specific steps of step S10 are as follows: Step S101: The state of the reinforcement learning agent, the state d at step t t =(e t ,e s ,r q )∈S, by the entity e at the tth step t , source entity e s and query relation r q Composition, S represents the set of all possible states; Step S102: Reinforce the actions of the learning agent, the action set A of step t t ={(r t ′,e t ′)|(e t ,r t ′,e t ′)∈G},A t ∈A, is the neighbor set of the current entity, which is composed of the outward edge of the current entity and the corresponding entity, where r t ′ represents the relationship selected by the agent at step t, e t ′ represents the next entity selected by the agent at step t, e t Indicates the entity where the agent is currently located at the tth step, G represents the entire knowledge graph, and the reasoning process is expressed as: where e s is the source entity, e T Indicates the entity at the last time step, r i (i=1…T) represents the relationship corresponding to the i-th time step; Step S103: Accuracy reward function, if the agent finally stays on the accurate entity, that is, e T =e o , e o represents the target entity, then the accuracy reward R s (S T )=R b (S T )=1, otherwise the pre-trained embedding model ConvE is used to calculate the probability f(e s ,r q ,e T ) as the accuracy reward, the specific form of the accuracy reward function is as follows: R s (s T )=R b (s T )+(1-R b (s T ))f(e s ,r q ,e T ) Where R b (S T ) indicates state S T The corresponding accuracy reward is (e s ,r q ,e T ) represents the triplet obtained by final reasoning; Step S104: Strategy function. The agent’s search strategy is parameterized by state information, query relationship information, and historical search path. Specifically, each entity and relationship in the knowledge graph is represented by embedding, and GRU is used to encode the historical path. The path encoding is in the following form: h 0 =GRU(0,a 0 ) h t =GRU(h t-1 ,a t-1 ),t>0 Where GRU(*) represents the gated recurrent unit, h 0 represents the initial path encoding state, a 0 is the default initial action, h t represents the path encoding state at time t, a t-1 represents the action chosen by the agent at time t-1; The parameterized policy network can be expressed as: π θ (to t+1 |s t )=softmax(A t W 2 ReLU(W 1 [h t ;and t ;r q ])) ReLU: represents a nonlinear activation function, that is, when the input is negative, the output is zero, and when the input is positive, the output is the input value; Softmax: represents the normalization function; W 1 Represents the weight matrix, which is used to encode the path h t Entity e t and query relation r q Combined into a vector; W 1 Represents the weight matrix, used to 1 The output of a is transformed t+1 A represents the action selected by the agent at time t+1; t represents the action space of the agent at time t, π θ (a t+1 |s t ) is in state s t Play A t The probability distribution of all actions in .

3. According to claim 1, the multi-hop knowledge graph reasoning method based on semantic and topological enhancement, Features: The specific steps of step S20 are as follows: Step S201: Calculate the local relation similarity LRS, and use the normalized cosine similarity to calculate the local relation similarity LRS of each relation r in the reasoning path. t Relationship with query q similarity to ensure that the relationships in the path maintain high semantic consistency; in r t ,r q The vector representation of , T is the total length of the relationship or entity sequence; Step S202: Calculate the global relationship closeness GRC by calculating each selected relationship r b The degree to which the distance between the target entity and the source entity is reduced measures the global semantic closeness of the relationship in the path; Cnt -Δ (r b ) indicates that the relationship r is selected in the path b The number of distances that decrease after that, Cnt +Δ (r b ) indicates that the relationship r is selected in the path b The number of distances that increase afterward; Step S203: Combining the local relationship similarity with the global relationship closeness to obtain the semantic consistency score RC of the relationship: Step S204: Calculate the local entity similarity LES, and calculate the t With the target entity e o To ensure that entities in the path maintain high semantic consistency: in Represents entity e t With the target entity e o The vector representation of ; Step S205: Calculate the global entity closeness GEC, by calculating the degree to which each selected entity reduces the distance between the target entity and the source entity, to measure the global semantic closeness of the entities in the path: Cnt -Δ (e b ) indicates the selection relationship e in the path b The number of distances that decrease after that, Cnt +Δ (e b ) indicates the selection relationship e in the path b The number of distances that increase afterward; Step S206: Combining the local entity similarity and the global entity closeness to obtain the entity's semantic consistency score EC: Step S207: The semantic consistency scores of the relationship and entity are integrated to obtain the semantic consistency score SC of the entire reasoning path: SC=RC×EC.

4. The multi-hop knowledge graph reasoning method based on semantic and topological enhancement according to claim 1, Features: The specific steps of step S30 are as follows: Step S301: construct an adjacency matrix, represent the entities and relationships in the knowledge graph as an adjacency matrix, and if there is a relationship between entities, the value of the corresponding position in the matrix is ​​increased by one; Step S302: Calculate the number of multi-hop paths, use the adjacency matrix to calculate the number of multi-hop relationship paths from each entity in the reasoning path to the target entity, and identify the key entities in the path; Step S303: Calculate the betweenness centrality BC. By counting the number of multi-hop paths, calculate the betweenness centrality of each entity in the reasoning path, and identify the key entities that cannot be replaced in the path reasoning process: in Represents the intermediate entity e t To the entity e at the last time step T The number of paths of length i, Represents all entities to the target entity e T The number of paths of length i, BC(e t ) measures the intermediate entity e t Importance in topology.

5. The multi-hop knowledge graph reasoning method based on semantic and topological enhancement according to claim 1, Features: The specific steps of step S40 are as follows: Step S401: Design accuracy reward function R S , integrating binary reward function and soft reward function; Binary reward function, which gives the agent a positive reward when the reasoning path is correct and a negative reward otherwise; The soft reward function uses the pre-trained embedding model to calculate the soft reward of the reasoning path, providing more refined feedback to the agent: R s (s T )=R b (s T )+(1-R b (s T ))×f(e s ,r q ,e T ) Where (1-R b (s T ))×f(e s ,r q ,e T ) is the soft reward function, R b (s T ) is a binary reward function; Step S402: Design rationality reward function R r , combining the semantic consistency constraint and the multi-hop betweenness centrality constraint, the rationality reward of the reasoning path is calculated: R r (r t ,e t )=αSC(r t ,e t )×(1-α)BC(e t ) Among them, α is the weight parameter, e t To calculate the entities in the reasoning path, SC(r t ,e t ) is through the comprehensive relationship r t and entity e t The semantic consistency score is calculated to measure the consistency of relations and entities in the path, BC(e t ) is calculated by calculating the entity e in the reasoning path t The betweenness centrality is used to identify key entities in the path; Step S403: The final reward function R is obtained by integrating the accuracy reward and the rationality reward. The binary reward, the soft reward and the rationality reward are integrated into the final reward function. By adjusting the relative importance weights of each part, the selection of the reasoning path is optimized to guide the agent to select the optimal path. R((r t ,e t ),s T )=R s (s T )×(1+R r (r t ,e t )) Step S404: The policy gradient optimization algorithm updates the policy network, and trains the policy network by maximizing the expected return of all queries in the knowledge graph through gradient ascent, where the cumulative expected return is: where π θ is the agent’s policy network, a 1 ,…a T The agent executes the query triple (e s ,r,e o ) process, The meaning is that all triples (e s ,r,e o )'s expected return, The meaning is that in strategy π θ A sequence of actions under the guidance of 1 ,…a T Expected value; The REINFORCE algorithm is used to solve the optimization problem and traverse all query triples in the knowledge graph (e s ,r,e o ) Update the parameters of the policy network by the following formula: Represents the policy network π θ The derivative of the parameter θ, R t represents the reward obtained by the agent at time t, logπ θ (a t |s t ) represents the policy network π θ In status t Take action a t The logarithmic probability of is the policy gradient approximated using REINFORCE, and θ′ is the updated parameter.

Images