Knowledge reasoning method and system based on agent dynamic path completion strategy

Abstract

The application belongs to the technical field of knowledge graph, and particularly relates to a knowledge reasoning method and system based on an agent dynamic path completion strategy, which extracts entities and the relationships between the entities in a target knowledge graph, and mines rules in the target knowledge graph and confidence score corresponding to the rules; a reinforcement learning agent is constructed, the agent is dynamically guided to complete the path of the knowledge graph according to the current entity state and historical path information through rules, and the total reward of the agent is calculated according to the hit reward when the agent hits a target entity and the rule reward when the reasoning path of the agent conforms to the rules, the strategy network of the agent is trained by maximizing the expected value of the total reward of the agent, and the corresponding knowledge reasoning result is obtained by using the trained agent to perform path reasoning in the knowledge graph for a given target condition to be queried. The application dynamically completes the most likely path in the reasoning process by using the dynamic path completion strategy to obtain a complete reasoning path, and solves the reasoning truncation problem caused by the missing path of a sparse knowledge graph.

Description

Technical Field

[0001] This invention belongs to the field of knowledge graph technology, and specifically relates to a knowledge reasoning method and system based on an agent dynamic path completion strategy. Background Technology

[0002] With the continuous development of information technology, artificial intelligence has evolved from "computational intelligence" (capable of storing and calculating) to "perceptual intelligence" (capable of hearing, speaking, seeing, and recognizing), and then to the next stage of "cognitive intelligence" (possessing the ability to understand, reason, and explain). The difficulty and value of achieving these three stages have increased progressively. Knowledge graphs are currently one of the core technologies of artificial intelligence. As a novel knowledge representation method, knowledge graphs contain a large amount of prior knowledge and organize massive amounts of information in the form of structured triples, linking and deeply integrating different data sources through entities and relationships. Currently, numerous knowledge graphs, such as YAGO, Dbpedia, and Freebase, have been developed, and related technologies have been widely applied in tasks such as intelligent question answering, recommendation systems, and information security, gaining widespread attention in both academia and industry.

[0003] Knowledge reasoning is the process of extracting new facts from known knowledge through reasoning and exploration, or of generalizing from individual pieces of knowledge to general knowledge by inductively applying existing knowledge. Early research on reasoning largely focused on logical description and knowledge engineering. Many scholars advocated using formal methods to describe the objective world, believing that all reasoning was based on existing logical knowledge, such as first-order logic and predicate logic. How to derive correct conclusions from known propositions and predicates remained a key focus of their research. In recent years, with the explosive growth of internet data, traditional methods based on manually built knowledge bases have become inadequate for the demands of mining vast amounts of knowledge in the era of big data. Therefore, data-driven reasoning methods have gradually become the mainstream in knowledge reasoning research.

[0004] Existing knowledge reasoning methods have achieved good results in experimental environments with sufficient sample sizes. However, in real-world scenarios, corpus samples typically follow a long-tail distribution, meaning most corpora are of low quality, resulting in sparse knowledge graphs. This sparsity leads to the loss of many key reasoning paths, frequently causing reasoning interruptions. Furthermore, the sparse graph environment makes it difficult for models to obtain sufficient training samples, lacking adequate information to guide model training and hindering the effective training of policy decision networks for reinforcement learning agents. Experiments show that many reasoning methods are unsuitable for the current sparse knowledge graph reasoning environment in real-world scenarios; their performance typically degrades significantly when applied to such environments. Summary of the Invention

[0005] To address this, the present invention provides a knowledge reasoning method and system based on an agent's dynamic path completion strategy, which solves the reasoning truncation problem caused by missing paths in sparse knowledge graphs. In a sparse knowledge graph environment, the method can dynamically complete the most likely path in the reasoning process through a dynamic path completion strategy to obtain a complete reasoning path, and then obtain the final result through reasoning.

[0006] According to the design scheme provided by this invention, a knowledge reasoning method based on an agent dynamic path completion strategy is provided, comprising the following:

[0007] Extract entities and relationships between entities from the target knowledge graph, and mine the rules and their corresponding confidence scores from the target knowledge graph;

[0008] Construct a reinforcement learning agent, dynamically guide the agent to complete the knowledge graph path according to the current entity state and historical path information through rules, and calculate the agent's total reward based on the hit reward when the agent hits the target entity and the rule reward when the agent's reasoning path conforms to the rules. Train the agent's policy network by maximizing the expected value of the agent's total reward.

[0009] Given a target condition to be queried, the trained agent obtains the corresponding knowledge reasoning result by performing path reasoning in the knowledge graph.

[0010] As a knowledge reasoning method based on the agent dynamic path completion strategy in this invention, further, in mining the rules in the target knowledge graph, a rule induction method is used to summarize structured chain rules in the target knowledge graph; and a preset score threshold is used to filter out rules with confidence scores higher than the score threshold.

[0011] As a knowledge reasoning method based on the agent's dynamic path completion strategy in this invention, the constructed reinforcement learning agent elements include: state, action, reward, policy network, and transition function. Here, the state represents the embedded representation of the current node of the agent, the action represents all possible next operations of the agent at the current node, the reward represents the feedback obtained by the agent after taking an action, the policy network represents the network in which the agent performs reinforcement learning based on the node state, action, reward, and transition function, and the transition function represents the transition result of the state after the agent's next action.

[0012] As a knowledge reasoning method based on an agent's dynamic path completion strategy in this invention, further, during the agent's path reasoning process, if the sequence of reasoning paths corresponds to a rule, the confidence level corresponding to the rule is used as an additional rule reward; and when the agent hits the correct tail entity, a hit reward is obtained; the total reward of the agent is represented as R.total =λR r +(1-λ)R h , where R h For the hit reward, R r The reward is determined by the rules, and λ is a preset weight parameter.

[0013] As a knowledge reasoning method based on the agent dynamic path completion strategy in this invention, when setting the agent rule reward, if two reasoning paths simultaneously satisfy multiple rules, the confidence scores of the multiple rules are compared, and the rule corresponding to the highest confidence score is selected as the path matching rule. The confidence score of the path matching rule is then used as the agent's rule reward.

[0014] As a knowledge reasoning method based on the agent dynamic path completion strategy of this invention, further, attention mechanism and rule guidance are combined and path completion algorithm is used to expand the agent action space and complete the knowledge graph path. In the path completion algorithm, first, the positive and negative sample set of the action space is set and the agent state vector and action vector are initialized. Then, the iterative expansion step of the agent action space and the iteration termination condition are set. In the iterative expansion step, the agent action space is updated by adding extra action space, and the agent's next action is obtained by the agent random sampling strategy function. The positive and negative sample set of the action space is updated by whether the agent reaches the target entity by walking. The iteration is terminated according to whether the termination condition is met.

[0015] As a knowledge reasoning method based on the agent dynamic path completion strategy of the present invention, the positive and negative sample sets of the action space are further updated by whether the agent reaches the target entity through walking. The update process is as follows: if the agent reaches the target entity through walking, and if the agent does not reach the target entity, the next action and the agent state are added to the negative sample set, and the agent policy function is resampled to obtain the agent's next action. The resampled agent's next action and state are added to the positive sample set. If the agent reaches the target entity through walking, the next action and agent state obtained before the iteration are added to the positive sample set.

[0016] As a knowledge reasoning method based on the agent dynamic path completion strategy of this invention, further, when updating the agent's action space, the probability of all possible paths in the path candidate set reasoned by the agent is measured by the probability of the agent's next action, and the attention weight of the relation in each path candidate set is calculated; according to the ranking of the attention weights in the path candidate set, the top x relations are selected as the set of most likely completion paths for the agent in the current state; according to the current entity e tThe complete path set, through link prediction based on the ConvE embedding method, provides the agent with an additional set of size N. add Action space It also combines the agent's original action space at the current node to update the agent's search space at nodes where the current inference path is missing. The update process is represented as

[0017] As a knowledge reasoning method based on the agent's dynamic path completion strategy in this invention, the probability calculation process of the agent's next action is further expressed as: P((r,e)∣s t )=P(r∣s t )P(e∣r,s t ), where P(r|s t P(e|r,s) represents the probability distribution of the agent choosing the next relation in the current state. t ) represents the probability distribution of the agent selecting the next entity. ; the attention weights of relations in each path candidate set are represented as in, Represents the identity matrix. represents the rule-guided auxiliary matrix, and r represents the candidate relations of the agent in the current state. This indicates the number of candidate relation elements.

[0018] Furthermore, the present invention also provides a knowledge reasoning system based on an agent-driven dynamic path completion strategy, comprising: a rule mining module, a path completion module, and a knowledge reasoning module, wherein,

[0019] The rule mining module is used to extract entities and relationships between entities from the target knowledge graph, and to mine the rules and their corresponding confidence scores in the target knowledge graph.

[0020] The path completion module is used to construct reinforcement learning agents. Based on the current entity state and historical path information, it dynamically guides the agents to complete knowledge graph paths through rules. It calculates the total reward of the agents based on the hit reward when the agents hit the target entity and the rule reward when the agents' reasoning path conforms to the rules. The agent's policy network is trained by maximizing the expected value of the total reward of the agents.

[0021] The knowledge reasoning module is used to obtain the corresponding knowledge reasoning results by using a trained agent to perform path reasoning in a knowledge graph for a given query target condition.

[0022] The beneficial effects of this invention are:

[0023] This invention utilizes dynamic path completion to achieve knowledge reasoning in sparse knowledge graph environments where inference paths are missing. It combines the sequential decision-making process of a reinforcement learning agent to dynamically complete missing paths in the reasoning process, achieving more comprehensive knowledge reasoning. Furthermore, it addresses the lack of information guidance during reasoning by incorporating both global and local knowledge. More efficient knowledge reasoning is achieved through iterative updates and mutual reinforcement of rule induction and factual reasoning. Compared to current neural networks that output inference results without interpretation, this application presents the reasoning process through interpretable paths, making the model more transparent and increasing trust in its decisions. The lack of interpretability of the reasoning method significantly impacts the reasoning results and related backtracking. Another advantage of the explicit reasoning scheme adopted in this application is the ability to better backtrack erroneous reasoning examples, facilitating practical applications. Attached image description:

[0024] Figure 1 This is a schematic diagram of the knowledge reasoning process in the embodiment;

[0025] Figure 2 This example illustrates knowledge reasoning in a sparse graph environment.

[0026] Figure 3 This is a schematic diagram of the agent's dynamic path completion strategy in the embodiment;

[0027] Figure 4 This is a schematic diagram of the iterative reasoning framework in the embodiment. Detailed implementation method:

[0028] To make the objectives, technical solutions, and advantages of this invention clearer and more understandable, the invention will be further described in detail below with reference to the accompanying drawings and technical solutions.

[0029] According to the embodiments of the present invention, see Figure 1 As shown, a knowledge reasoning method based on an agent's dynamic path completion strategy is provided, comprising:

[0030] S101. Extract entities and relationships between entities from the target knowledge graph, and mine the rules and confidence scores corresponding to the rules in the target knowledge graph;

[0031] S102. Construct a reinforcement learning agent. Based on the current entity state and historical path information, dynamically guide the agent to complete the knowledge graph path through rules. Calculate the agent's total reward based on the hit reward when the agent hits the target entity and the rule reward when the agent's reasoning path conforms to the rules. Train the agent's policy network by maximizing the expected value of the agent's total reward.

[0032] S103. Given the target conditions to be queried, the trained agent obtains the corresponding knowledge reasoning results by performing path reasoning in the knowledge graph.

[0033] like Figure 2 As shown, the sparsity of the knowledge graph environment leads to the loss of many key reasoning paths. Therefore, knowledge reasoning in a sparse knowledge graph environment often results in reasoning interruptions. Simultaneously, the sparse knowledge graph environment makes it difficult for the model to obtain sufficient training samples, lacking adequate information to guide model training and hindering the effective training of the policy decision network of the reinforcement learning agent. In this embodiment, dynamic path completion is used to achieve complete path reasoning, solving the reasoning truncation problem caused by missing paths in the sparse knowledge graph.

[0034] Furthermore, in mining the rules within the target knowledge graph, structured chain rules are summarized using rule induction methods. A preset score threshold is then used to filter out rules with confidence scores higher than the threshold. The constructed reinforcement learning agent includes: state, action, reward, policy network, and transition function. Here, state represents the embedding representation of the agent's current node, action represents all possible next actions the agent can take at the current node, reward represents the feedback received after taking an action, policy network represents the network through which the agent performs reinforcement learning based on node state, action, reward, and transition function, and transition function represents the transition result after the agent takes the next action.

[0035] This study employs reinforcement learning for knowledge reasoning, modeling the multi-hop reasoning problem as a sequential decision-making problem. A reinforcement learning agent is trained through feedback and interaction. During path exploration, the agent learns paths that yield high rewards to update its policy network, and then uses reinforcement learning for path reasoning. When a reasoning path is interrupted, resulting in missing paths, the reinforcement learning agent dynamically completes the path based on the current entity's state and historical path information. The action space is expanded by adding additional actions based on historical information (encoded via LSTM) and current state information. Figure 3 As shown, during the dynamic path completion process, when constructing new additional action spaces, the reinforcement learning agent dynamically expands the path by combining global information, thereby alleviating the problem of path discontinuity in multi-hop inference.

[0036] In a preferred embodiment, further, during the agent's path reasoning process, if the sequence of the reasoning path corresponds to a rule, the confidence level corresponding to the rule is used as an additional rule reward; and when the agent hits the correct tail entity, a hit reward is obtained; the total reward of the agent is represented as R.total =λR r +(1-λ)R h , where R h For the hit reward, R r The reward is determined by the rules, and λ is a preset weight parameter.

[0037] The AnyBURL rule-based induction method extracts rules from the knowledge graph. These rules are frequently occurring path combination patterns in the graph, providing global information guidance for the path reasoning of reinforcement learning agents. Examples of rule extraction are as follows:

[0038] concept:athlete_playsin_league(a,b)←concept:athlete_playsin_team(a,e)∧concept:team_playsin_league(e,b)

[0039] Each rule corresponds to a confidence score. The higher the confidence score, the more credible the rule is. A rule with a higher confidence score indicates that the pattern appears more frequently in the knowledge graph, and therefore the rule is more credible.

[0040]

[0041] In one rule, the rule header is r(...), and the rule's subscripts are b1(...),...,b n (...). Therefore, a rule It can be represented as The uppercase letters represent variables, r(c i ,c j ) and the triplet in the spectrum (c i ,r,c j )equivalence.

[0042] To dynamically guide the agent's path search through rules, this invention provides additional rule rewards for paths that conform to the rules during the agent's path exploration process. If the sequence of the agent's reasoning paths corresponds to a rule, i.e., b1 = r... j-i+1 ,...,b i-1 =r j-1 We use the confidence level corresponding to the rule as an additional rule reward c. ij ∈£. The higher the credibility of the rule corresponding to the reasoning path, the higher the rule reward.

[0043]

[0044] If the reasoning path explored by the reinforcement learning agent simultaneously satisfies the rules and rules At this point, the confidence scores of R1 and R2 are compared, and the rule corresponding to the larger confidence score is taken as the matching rule.

[0045] As a preferred embodiment, further, attention mechanism and rule guidance are combined with path completion algorithm to expand the agent's action space and complete the knowledge graph path. In the path completion algorithm, firstly, a set of positive and negative samples in the action space is set and the agent's state vector and action vector are initialized. Then, an iterative expansion step and an iteration termination condition are set for the agent's action space. In the iterative expansion step, the agent's action space is updated by adding additional action space, and the agent's next action is obtained through the agent's random sampling strategy function. The set of positive and negative samples in the action space is updated by whether the agent reaches the target entity through walking. The iteration is terminated based on whether the termination condition is met.

[0046] By combining attention mechanisms and rule-based guidance to complete high-quality paths, and then further expanding the action space of the reinforcement learning agent, the probability calculation method for the agent's next action is as follows:

[0047] P((r,e)∣s t )=P(r∣s t )P(e∣r,s t ).

[0048] The attention weights for each relation in the candidate set are calculated as follows, by dynamically measuring the probability of all possible paths in the candidate set explored by the reinforcement learning agent:

[0049]

[0050] Based on the ranking of weights w in the candidate path set, the top x relations are selected as the set of paths most likely to be completed by the agent in the current state. Next, based on the current entity e... t The path, along with the dynamically completed path mentioned above, is used for link prediction via the embedding-based ConvE method, providing an additional link of size N for the reinforcement learning agent. add The action space, i.e., N add =kx. Combining the agent's original action space at the current node, the agent generates a larger search space at nodes where the current inference path is missing. Right now

[0051] By employing the dynamic path completion strategy proposed above, when the reinforcement learning agent experiences an interruption in its inference path at the current node, it can dynamically complete the inference path by combining global information provided by the rules and target prediction information, and then proceed to the next inference jump. This method provides the reinforcement learning agent with a larger and more efficient action search space, thereby alleviating the inference truncation problem caused by missing edges in the sparse knowledge graph. The specific implementation algorithm can be designed as follows:

[0052]

[0053]

[0054] In the algorithm, the positive and negative sample sets in the action space are updated based on whether the agent reaches the target entity through roaming. If the agent reaches the target entity through roaming, and if not, the next action and agent state are added to the negative sample set. The agent's policy function is then resampled to obtain the next action, and the resampled next action and state are added to the positive sample set. If the agent reaches the target entity through roaming, the next action and agent state obtained before the iteration are added to the positive sample set. The steps of the algorithm can be summarized as follows:

[0055] I. Initialize the embedded representation of entities and relations in the knowledge graph.

[0056] II. Use the rule-based induction method AnyBURL to mine the rules and their corresponding confidence scores in the graph.

[0057] III. Utilize reinforcement learning agents to explore paths in the graph. When the agent encounters path interruption due to the sparse graph environment, use a dynamic path completion algorithm to expand the agent's action space.

[0058] IV. Calculate the total reward for the reinforcement learning agent. The reward consists of two parts: a hit reward and a rule reward. When the agent hits a target entity, it receives a hit reward R. h When the agent's reasoning path conforms to a combination of rules, the agent receives the rule reward R. r The final reward obtained by the agent is the sum of the two partial rewards, namely: R total =λR r +(1-λ)R h .

[0059] V. The agent's policy network is trained by maximizing the expected value J(θ) of the sum of rewards, where the agent's parameters are updated by maximizing the expected value of the sum of rewards using the REINFORCE algorithm.

[0060]

[0061]

[0062]

[0063] VI. By using an iterative reasoning framework, the rule induction module and the factual reasoning module are iteratively trained to achieve more efficient knowledge reasoning.

[0064] See Figure 4 As shown, an iterative reasoning framework is used to address the lack of information guidance for the agent during reasoning by combining rule induction and factual reasoning. In rule induction, the AnyBEURL model is used to inductively summarize structured chain rules in the knowledge graph and filter rules with confidence levels above a given threshold. These high-confidence rules represent frequent path patterns in the graph, thus summarizing some global graph information and providing rule rewards and global information guidance for further training and reasoning of the reinforcement learning agent. In factual reasoning, reinforcement learning combined with a dynamic path completion strategy is used for factual reasoning. The reinforcement learning agent obtains a hit reward R by hitting the correct tail entity. h Furthermore, during the exploration path in the reasoning process, paths that conform to the rules will receive a greater rule reward R. r The agent is rewarded with R based on hitting the target entity. h and rule rewards R r By employing the REINFORCE algorithm in reinforcement learning, the total reward obtained by the agent is maximized, and the agent's parameters are updated. This encourages the agent to explore high-reward paths, meaning it is more inclined to reason about paths that hit the correct entity and paths that conform to high-confidence rules. Through repeated exploration of knowledge graph paths, the trained and converged reinforcement learning agent, combined with a dynamic path completion strategy, infers new fact triples in the current knowledge graph environment.

[0065] Rule induction and fact reasoning employ a mutually reinforcing optimization strategy through iterative training. Rule induction provides more high-quality rules for fact reasoning, guiding the agent to perform more efficient path search and achieve better link prediction results. Fact reasoning, combined with the rules provided by rule induction, yields more accurate fact triples. More fact triples help the rule induction module to infer more high-confidence rules, creating a mutually reinforcing optimization effect. Through this iterative reasoning framework, rule induction and fact reasoning iteratively learn and reason, generating more high-quality fact triples and high-confidence rules, achieving a mutually reinforcing optimization effect. The reinforcement learning agent can perform more fine-grained reasoning tasks under the guidance of global information, to some extent compensating for the information deficiency caused by sparse graphs.

[0066] Furthermore, based on the above method, this embodiment of the invention also provides a knowledge reasoning system based on an agent-based dynamic path completion strategy, comprising: a rule mining module, a path completion module, and a knowledge reasoning module, wherein...

[0067] The rule mining module is used to extract entities and relationships between entities from the target knowledge graph, and to mine the rules and their corresponding confidence scores in the target knowledge graph.

[0068] The path completion module is used to construct reinforcement learning agents. Based on the current entity state and historical path information, it dynamically guides the agents to complete knowledge graph paths through rules. It calculates the total reward of the agents based on the hit reward when the agents hit the target entity and the rule reward when the agents' reasoning path conforms to the rules. The agent's policy network is trained by maximizing the expected value of the total reward of the agents.

[0069] The knowledge reasoning module is used to obtain the corresponding knowledge reasoning results by using a trained agent to perform path reasoning in a knowledge graph for a given query target condition.

[0070] Unless otherwise specifically stated, the relative steps, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0071] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0072] The units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations are not considered to be beyond the scope of this invention.

[0073] Those skilled in the art will understand that all or part of the steps in the above methods can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Optionally, all or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiments can be implemented in hardware or as a software functional module. This invention is not limited to any particular combination of hardware and software.

[0074] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A knowledge reasoning method based on an agent's dynamic path completion strategy, characterized in that, Includes the following content: Extract entities and relationships between entities from the target knowledge graph, and mine the rules and confidence scores corresponding to the rules from the target knowledge graph. The target knowledge graph is a knowledge graph constructed from intelligent question answering corpus samples. A reinforcement learning agent is constructed. Based on the current entity state and historical path information, the agent is dynamically guided to complete knowledge graph paths according to rules. The total reward of the agent is calculated based on the hit reward when the agent hits the target entity and the rule reward when the agent's inference path conforms to the rule. The policy network of the agent is trained by maximizing the expected value of the total reward. During the agent's path reasoning process, if the sequence of reasoning paths corresponds to a rule, the confidence level corresponding to the rule is used as an additional rule reward; and a hit reward is obtained when the agent hits the correct tail entity. The total reward of the agent is represented as R. total =λR r +(1-λ)R h R h For the hit reward, R r The reward is based on the rules, and λ is a preset weight parameter. For a given target condition consisting of entities and relationships, the trained agent uses path reasoning in the intelligent question-answering knowledge graph to obtain the target entity of the target condition and uses the target entity as the knowledge reasoning result.

2. The knowledge reasoning method based on the agent dynamic path completion strategy according to claim 1, characterized in that, In mining rules from the target knowledge graph, structured chain rules are summarized in the target knowledge graph using rule induction methods; and rules with confidence scores higher than the preset score threshold are selected using preset score thresholds.

3. The knowledge reasoning method based on the agent dynamic path completion strategy according to claim 1, characterized in that, The elements of the constructed reinforcement learning agent include: state, action, reward, policy network, and transition function. The state represents the embedding representation of the current node of the agent, the action represents all possible next actions of the agent at the current node, the reward represents the feedback obtained by the agent after taking an action, the policy network represents the network by which the agent performs reinforcement learning based on the node state, action, reward, and transition function, and the transition function represents the transition result of the state after the agent's next action.

4. The knowledge reasoning method based on the agent dynamic path completion strategy according to claim 1, characterized in that, When setting the rule reward for the agent, if two inference paths satisfy multiple rules at the same time, the confidence scores of the multiple rules are compared, and the rule corresponding to the highest confidence score is selected as the path matching rule. The confidence score of the path matching rule is then used as the rule reward for the agent.

5. The knowledge reasoning method based on the agent dynamic path completion strategy according to claim 1 or 4, characterized in that, This paper combines attention mechanisms and rule guidance with path completion algorithms to expand the agent's action space and complete knowledge graph paths. In the path completion algorithm, firstly, a set of positive and negative samples in the action space is set and the agent's state vector and action vector are initialized. Then, an iterative expansion step and an iteration termination condition are set for the agent's action space. In the iterative expansion step, the agent's action space is updated by adding additional action space, and the agent's next action is obtained through a random sampling policy function. The set of positive and negative samples in the action space is updated based on whether the agent reaches the target entity through walking. The iteration is terminated based on whether the termination condition is met.

6. The knowledge reasoning method based on the agent dynamic path completion strategy according to claim 5, characterized in that, The positive and negative sample sets in the action space are updated based on whether the agent reaches the target entity through walking. The update process is as follows: If the agent reaches the target entity through walking, and if it does not reach the target entity, the next action and the agent state are added to the negative sample set, and the agent policy function is resampled to obtain the agent's next action. The resampled agent next action and state are added to the positive sample set. If the agent reaches the target entity through walking, the next action and agent state obtained before the iteration are added to the positive sample set.

7. The knowledge reasoning method based on the agent dynamic path completion strategy according to claim 5, characterized in that, When updating the agent's action space, the probability of the agent's next action is used to measure the possible probabilities of all paths in the candidate path set in the agent's reasoning, and the attention weight of each relation in the candidate path set is calculated. Based on the ranking of the attention weights in the candidate path sets, the top x relations are selected as the set of the most likely completion paths for the agent in the current state. Based on the current entity e... t The complete path set, through link prediction based on the ConvE embedding method, provides the agent with an additional set of size N. add Action space It also combines the agent's original action space at the current node to update the agent's search space at nodes where the current inference path is missing. The update process is represented as 8. The knowledge reasoning method based on the agent dynamic path completion strategy according to claim 6, characterized in that, The probability calculation process for the agent's next action is expressed as: P((r,e)∣s t )=P(r∣s t )P(e∣r,s t ), where s t Let P(r|s) represent the current entity, r represent the relationship the agent chooses for the next step, and e represent the entity in the graph. t P(e|r,s) represents the probability distribution of the agent choosing the next relation in the current state. t ) represents the probability distribution of the agent choosing the next entity; the attention weight of the relation in each path candidate set is represented as in, Represents the identity matrix. represents the auxiliary matrix that guides the rule body, and r represents the candidate relation of the agent in the current state. This indicates the number of candidate relation elements.

9. A knowledge reasoning system based on an agent-driven dynamic path completion strategy, characterized in that, It includes: a rule mining module, a path completion module, and a knowledge reasoning module. The rule mining module is used to extract entities and relationships between entities from the target knowledge graph, and to mine rules and their corresponding confidence scores from the target knowledge graph. The target knowledge graph is a knowledge graph constructed from intelligent question-answering corpus samples. The path completion module is used to construct the reinforcement learning agent. Based on the current entity state and historical path information, it dynamically guides the agent to complete knowledge graph paths according to rules. The total agent reward is calculated based on the hit reward when the agent hits the target entity and the rule reward when the agent's inference path conforms to the rules. The agent's policy network is trained by maximizing the expected value of the total agent reward. During the agent's path reasoning process, if the sequence of inference paths corresponds to a rule, the confidence level corresponding to the rule is used as an additional rule reward; and a hit reward is obtained when the agent hits the correct tail entity. The total agent reward is represented as R. total =λR r +(1-λ)R h R h For the hit reward, R r The reward is based on the rules, and λ is a preset weight parameter. The knowledge reasoning module is used to obtain the target entity corresponding to the query target condition composed of given entities and relationships by using a trained agent to perform path reasoning in the intelligent question answering knowledge graph, and then use the target entity as the knowledge reasoning result.

Images