Sequence generation type knowledge reasoning method and system based on decision transformer

Abstract

The application belongs to the technical field of knowledge graphs, and particularly relates to a sequence generation type knowledge reasoning method and system based on a Decision Transformer, which extracts entities and relationships between entities in a target knowledge graph, and induces a structured rule sequence in the target knowledge graph; a Decision Transformer reasoning model for knowledge reasoning and dynamic interaction with a graph environment is constructed, wherein the Decision Transformer reasoning model adopts a rule guiding method to construct training path samples of the reasoning model when dynamically interacting with the graph environment, and utilizes the training path samples to train the Decision Transformer reasoning model to learn co-occurrence ability in the rules in the path; knowledge reasoning is performed through the Decision Transformer reasoning model based on the extracted entities and relationships between entities and the rule sequence, and a reasoning result is output. Through the sequence generation type reasoning process of dynamic interaction with the graph environment, the Decision Transformer architecture is utilized to process the reasoning sequence in parallel according to the current reasoning state, and the next action is reasoned in a generation type, so that faster knowledge reasoning is achieved, and reasoning efficiency is improved and reasoning interpretability is enhanced.

Description

Technical Field

[0001] This invention belongs to the field of knowledge graph technology, and specifically relates to a sequence-generative knowledge reasoning method and system based on Decision Transformer. Background Technology

[0002] With the widespread application of artificial intelligence (AI) technology, the interpretability of AI has received increasing attention. In currently popular deep learning models, the complex processing mechanisms and large number of parameters make it difficult for humans to trace and understand their reasoning processes. This results in poor interpretability for this end-to-end black-box learning method. High-performance, complex algorithms, models, and systems generally cannot explain their autonomous decisions and behaviors to human users, lacking interpretability of their decision-making logic. Interpretability is crucial for users to effectively understand, trust, and manage AI applications, contrasting sharply with the "black box" concept in deep neural networks. Uninterpretable models often fail to predict correct results in practice. While this may not have serious consequences in low-risk environments (e.g., video recommendation systems), it is dangerous for systems with high reliability requirements (e.g., medical, legal, and information security fields), where the model must explain how it arrives at its predictions. Therefore, achieving Explainable Artificial Intelligence (XAI) is essential. It provides a foundation of trust, allowing AI to function more broadly, helping to identify potential errors, improve models, enhance the quality of information services, meet ethical and legal requirements, and provide users with smarter services. Knowledge graphs are one of the core technologies of artificial intelligence. As a novel knowledge representation method, knowledge graphs contain a large amount of prior knowledge and organize massive amounts of information in the form of structured triples. They connect and deeply integrate different data sources through entities and relations. As a semantic and structured expression of knowledge, knowledge graphs perform knowledge reasoning by expressing entities and relations in a human-understandable form. Through auxiliary means such as reasoning paths and logical rules, combined with the entity information surrounding nodes, they enable explicit and interpretable knowledge graph reasoning, providing a solution for achieving interpretable artificial intelligence. Currently, numerous knowledge graphs, such as YAGO, Dbpedia, and Freebase, have been developed and have broad application prospects in information retrieval, information security, and cyberspace security, attracting widespread attention from both academia and industry.

[0003] Knowledge reasoning is the process of extracting new facts from known knowledge through reasoning and mining, or generalizing from individual knowledge to general knowledge by summarizing a large amount of existing knowledge. Early research on reasoning largely focused on logical description and knowledge engineering. Many scholars advocated using formal methods to describe the objective world, believing that all reasoning is based on existing logical knowledge, such as first-order logic and predicate logic. How to derive correct conclusions from known propositions and predicates has always been a key focus of their research. In recent years, with the explosive growth of internet data, traditional methods based on manually built knowledge bases have become inadequate for the demands of mining massive amounts of knowledge in the era of big data. Currently, knowledge reasoning oriented towards knowledge graphs is a typical representative of the field. This type of method combines concepts, attributes, and relationships on the graph structure of knowledge graphs, using relevant reasoning techniques to perform specific related knowledge reasoning tasks. The concepts, attributes, and relationships contained in knowledge graphs are naturally explanatory and more in line with human cognition of explanation, facilitating intuitive modeling of real-world reasoning and explanation scenarios. While current knowledge graph-based reasoning methods have achieved good results and provided some interpretability, they still have the following drawbacks: (1) During the reasoning process, the reasoning agent needs to explore a large number of paths, and each additional hop in the reasoning process causes the search space to grow exponentially. Therefore, current reasoning models usually require a long convergence time and have a slow reasoning speed. (2) In real-world environments, corpora are mostly low-resource, so the constructed knowledge graphs are mostly sparse. The sparseness of the graph environment causes many key reasoning paths to be missing, which can lead to reasoning interruptions in the reasoning module. Summary of the Invention

[0004] To address this, the present invention provides a sequence-generative knowledge reasoning method and system based on Decision Transformer, which enables dynamic interaction between the knowledge graph and the reasoning process, solves the interruption phenomenon caused by missing paths during the reasoning process, and facilitates application in real-world scenarios.

[0005] According to the design scheme provided by this invention, a sequence-generative knowledge reasoning method based on Decision Transformer is provided, comprising the following:

[0006] Extract entities and relationships between entities from the target knowledge graph, and summarize a structured sequence of rules in the target knowledge graph;

[0007] A Decision Transformer reasoning model is constructed for knowledge reasoning and dynamic interaction with the graph environment. When the Decision Transformer reasoning model interacts dynamically with the graph environment, it adopts a rule-guided approach to construct training path samples for the reasoning model and uses these training path samples to train the Decision Transformer reasoning model to learn the co-occurrence ability of rules in the path.

[0008] Based on the extracted entities, relationships between entities, and rule sequences, knowledge reasoning is performed using the Decision Transformer reasoning model, and the reasoning results are output.

[0009] As a sequence-generative knowledge reasoning method based on Decision Transformer in this invention, further, in mining the rules in the target knowledge graph, a rule induction method is used to summarize structured rules in the target knowledge graph; and a preset score threshold is used to filter out rules with confidence scores higher than the score threshold.

[0010] As a sequence-generative knowledge reasoning method based on Decision Transformer in this invention, the Decision Transformer reasoning model further includes an encoder and a decoder. In the encoder, each entity and relation is embedded to generate a token sequence, and the order of reasoning processes is distinguished by adding position encoding to the embedded representation of the entity or relation. In the decoder, the next reasoning action is output by performing task decoding on the sequence preceding the current learning sequence item.

[0011] As a sequence-generative knowledge reasoning method based on Decision Transformer in this invention, the decoder of the reasoning model further uses the Causal Transformer model to decode the token sequence of the encoder and generate the next reasoning sequence.

[0012] As a sequence-generative knowledge reasoning method based on Decision Transformer in this invention, the Decision Transformer reasoning model further employs cross-entropy as the target loss function and uses label smoothing to smooth the loss function. The target loss function is expressed as: Where τ represents the sequence, k represents the sequence index, V represents the dictionary consisting of the elements in the sequence, and α i The weights represent the smoothness of the labels, q represents the query, and p(i|q,τ) represents the weights of the labels. ＜k) represents the probability distribution of the sequence generated based on the current sequence and the next step of the query, ∈ is a set hyperparameter, and K represents the total number of sequence categories.

[0013] As a sequence-generative knowledge reasoning method based on Decision Transformer in this invention, further, in the training path samples of the reasoning model constructed by rule-guided method, firstly, all k random walk paths under the preset query conditions are traversed by random walk method; then, the entity and relation representations in the graph are initialized by using graph embedding method, and each path of all entity and relation element pairs in the random walk is traversed; then, all random walk paths are ranked in importance according to the path rule reward and the semantic similarity between the path and the query conditions, and the top N paths are selected as training path samples.

[0014] As a sequence-generative knowledge reasoning method based on Decision Transformer in this invention, further, in ranking the importance of all random walk paths according to the rule rewards of the path and the semantic similarity between the path and the query conditions, the following formula is used: To calculate path scores, importance is ranked using the score rankings, where p i Represents path i, a i c represents the semantic similarity between the corresponding path and the query conditions. i This represents the rule reward for the corresponding path, where λ is the weight parameter.

[0015] As a sequence-generative knowledge reasoning method based on Decision Transformer in this invention, further, the semantic similarity α between the path and the query conditions... i The value is calculated by using path embedding representation and the attention value of preset query conditions. The calculation process is represented as: a i =softmax(q) T r i ), Where, r i For all embedding representations of the corresponding path samples, p ij Let q represent the entity and relation element pairs corresponding to this path sequence, q represent the query conditions, and A represent the weight matrix composed of the corresponding attention values.

[0016] As a sequence-generative knowledge reasoning method based on Decision Transformer in this invention, the rule reward of the path is obtained through the confidence score of the rule. If the random walk path sequence corresponds to the extracted rule, the confidence score corresponding to the rule is used as the rule reward. If the random walk path satisfies multiple rules at the same time, the rule with the highest confidence score is selected as the matching rule, and the confidence score corresponding to the matching rule is used as the rule reward.

[0017] Furthermore, this invention also provides a sequence-generative knowledge reasoning system based on Decision Transformer, comprising: a rule mining module, a model building module, and a knowledge reasoning module, wherein,

[0018] The rule mining module is used to extract entities and relationships between entities from the target knowledge graph and to summarize structured rule sequences in the target knowledge graph.

[0019] The model building module is used to build a DecisionTransformer reasoning model for knowledge reasoning and dynamic interaction with the graph environment. When the DecisionTransformer reasoning model interacts dynamically with the graph environment, it uses a rule-guided approach to build training path samples for the reasoning model and uses these training path samples to train the DecisionTransformer reasoning model to learn the co-occurrence ability of rules in the path.

[0020] The knowledge reasoning module is used to perform knowledge reasoning based on the extracted entities, relationships between entities, and rule sequences, using the Decision Transformer reasoning model, and output the reasoning results.

[0021] The beneficial effects of this invention are:

[0022] This invention employs sequence-generative knowledge reasoning to achieve rapid knowledge reasoning for knowledge graphs. By modeling the entire knowledge reasoning process as a sequence, and using a Decision Transformer with an encoder-decoder structure to model this sequence, the next reasoning sequence is generated autoregressively, achieving more efficient and faster knowledge reasoning. Furthermore, thanks to the efficiency advantage of sequence-generative reasoning, this solution can be adapted to larger-scale real-world knowledge graphs for reasoning applications. Leveraging the advantages of the sequence generation model, the reasoning model in this invention no longer learns through path exploration but instead achieves path reasoning through sequence generation, making the model more robust to missing paths in the reasoning process. Compared to current black-box neural networks that directly output reasoning results without interpretation, this solution can demonstrate the reasoning process through interpretable paths, making the model more transparent and increasing user trust in the model's decisions. Moreover, the uninterpretability of the reasoning method significantly impacts the reasoning results and related backtracking; another benefit of the explicit reasoning used in this solution is the ability to better backtrack erroneous reasoning examples. Attached image description:

[0023] Figure 1 This is a schematic diagram of the knowledge reasoning process in the embodiment;

[0024] Figure 2 This is a schematic diagram of the sequential knowledge reasoning process in the embodiment;

[0025] Figure 3 This is a schematic diagram of the Decision Transformer inference model in the embodiment;

[0026] Figure 4 This is a schematic diagram of the rule-guided path sample construction process in the embodiment. Detailed implementation method:

[0027] To make the objectives, technical solutions, and advantages of this invention clearer and more understandable, the invention will be further described in detail below with reference to the accompanying drawings and technical solutions.

[0028] According to the embodiments of the present invention, see Figure 1 As shown, a sequence-generative knowledge reasoning method based on Decision Transformer is provided, including:

[0029] S101. Extract entities and relationships between entities from the target knowledge graph, and summarize a structured rule sequence in the target knowledge graph;

[0030] S102. Construct a Decision Transformer reasoning model for knowledge reasoning and dynamic interaction with the graph environment. When the Decision Transformer reasoning model interacts dynamically with the graph environment, it uses a rule-guided approach to construct training path samples for the reasoning model and uses these training path samples to train the Decision Transformer reasoning model to learn the co-occurrence ability of rules in the path.

[0031] S103. Based on the extracted entities, relationships between entities, and rule sequences, knowledge reasoning is performed using the DecisionTransformer reasoning model, and the reasoning results are output.

[0032] Decision Transformer treats Regression (RL) as an autoregressive sequence modeling problem, modeling the relationships between return-to-go sequences, state sequences, and action sequences. Compared to the commonly accepted concept of behavior cloning, which only models state-action relationships, it additionally considers the return and past triple (Rt, St, At) sequences. Figure 2 As shown, unlike previous reinforcement learning approaches based on trial and error, this embodiment utilizes a Decision Transformer inference model to simulate the interaction process of reinforcement learning. The inference module learns from the rewards, states, and action inputs of historical sequences and then generatively outputs the next inference action. Therefore, during training, the inference model in this solution can process inference sequences in parallel using the Transformer architecture. The inference module and the knowledge graph environment interact dynamically, and the inference process generatively infers the next inference sequence based on the current inference state, achieving faster inference.

[0033] In mining rules from a target knowledge graph, rule induction methods can be used to summarize structured rules within the graph; and a preset score threshold can be used to filter out rules with confidence scores higher than the threshold. In modeling the sequential knowledge reasoning process, i.e., given a head entity e... s Given query q, the model generates a complete inference process sequence τ, with the tail entity e of this sequence. o That is, the result of the reasoning, expressed as: q:=(e s ,r q ),τ:=(R1,a1,s1,...,R n ,a n ,e o ,<eos>).

[0034] As a preferred embodiment, the Decision Transformer inference model further includes an encoder and a decoder. In the encoder, each entity and relation is embedded to generate a token sequence, and positional encoding is added to the embedded representations of entities or relations to distinguish the order of inference processes. In the decoder, the next inference action is output by decoding the sequence preceding the currently learned sequence item. The decoder of the inference model uses the CausalTransformer model to decode the encoder's token sequence and generate the next inference sequence.

[0035] See Figure 3 As shown, in the encoder module of the Transformer, each entity and relation can be embedded using the knowledge graph embedding model ConvE, mapping each token to a vector of the same dimension. Simultaneously, to further distinguish the order of reasoning, corresponding positional encodings can be added to the embedding representation of each entity or relation. In the decoder module, the Causal Transformer model can be used to decode and generate the embedded sequence. By learning the sequence patterns in the reasoning path, it autoregressively generates the next possible action. The original Transformer inputs the entire context into the model and learns some co-occurrence patterns in the sequence through a self-attention mechanism. However, when applied to reasoning tasks, this mechanism exposes subsequent reasoning sequences to the model for learning, failing to simulate the purpose of reasoning. In this embodiment, the Causal Transformer can be used, inputting only the sequence preceding the currently learned sequence item into the model and outputting the next action, thus making it more suitable for knowledge reasoning tasks.

[0036]

[0037] p(·∣q,τ ＜k )=CausalTransformer(Encoded_Sequence(e s ,r q ,τ ＜k ))

[0038] During training, cross-entropy can be used as the loss function, and label smoothing can be applied to smooth the loss function to prevent overfitting. The specific loss function can be expressed as follows:

[0039]

[0040]

[0041] As a preferred embodiment, further, in constructing the training path samples of the inference model using a rule-guided approach, firstly, all k or fewer random walk paths under the preset query conditions are traversed using a random walk method; then, the entity and relation representations in the graph are initialized using a graph embedding method, and each path is traversed by all entity and relation element pairs in the random walk path; then, all random walk paths are ranked by importance based on path rule rewards and semantic similarity between the path and the query conditions, and the top N paths are selected as training path samples.

[0042] By generating high-quality path training samples, this invention guides the Dicision Transformer to learn the ability to co-occur sequences within paths. Simply relying on random walk strategies to sample paths may result in low-quality samples or even introduce noise. To construct high-quality path samples and enable the sequence generation model to learn better, this invention employs a rule-guided approach to guide the construction of training path samples. Rules are frequently occurring path combination patterns in the graph, providing global information to guide the selection of high-quality training samples. By extracting rules with global information and combining them with attention mechanisms and rule-based global information guidance, the paths searched by random walks become more representative.

[0043] like Figure 4 As shown, given a training sample query, we first traverse all random walk paths within k hops for that query using a random walk method. Then, we initialize the entity and relation representations in the knowledge graph using a knowledge graph embedding method, and for each path p, we iterate through all elements in the random walk paths. i To express.

[0044]

[0045] Where r i p represents the corresponding path sample. ij This refers to each element (entity and relation) in the path sequence.

[0046] Next, in the stage of evaluating the quality of path samples, the embeddings r of all paths are calculated. i The semantic similarity between each path and its corresponding query is calculated using the attention value of query q. The calculation method is as follows:

[0047] a i =softmax(q) T r i )

[0048] During the rule-guided phase, rules can be extracted from the knowledge graph using the AnyBURL rule induction method. These rules are frequently occurring path combination patterns in the graph, providing global information guidance for the reinforcement learning agent's path reasoning. An example of rule extraction is shown below:

[0049] concept:athlete_playsin_league(a,b)←concept:athlete_playsin_team(a,e)∧concept:team_playsin_league(e,b)

[0050] Each rule corresponds to a confidence score; a higher confidence score indicates a higher level of credibility for the corresponding rule. Furthermore, rules with higher confidence scores indicate that the pattern appears more frequently in the knowledge graph, thus increasing the rule's credibility. Additional rule rewards are applied to the random walk paths based on the high-confidence rules. If the random walk path sequence corresponds to an extracted rule, the confidence score of that rule is used as an additional rule reward c. i The higher the credibility of the rule corresponding to the reasoning path, the higher the confidence of the rule will be, and the higher the rule reward obtained by the sample of that path will be.

[0051]

[0052] At the same time, if the path of the random walk satisfies the rules and rules At this point, the confidence scores of R1 and R2 are compared, and the rule corresponding to the larger confidence score is taken as the matching rule.

[0053] Taking into account both the attention score and the rule reward score based on semantic similarity, the final score is calculated. The importance of all random walk paths is ranked by score, and the top N paths are selected as high-quality path samples. These path samples are more reliable than training paths obtained through simple random walks, thus better training the Dicision Transformer inference module to learn the ability to learn sequence co-occurrence within paths. The specific algorithm can be designed as follows:

[0054]

[0055]

[0056] First, the embedded representations of entities and relations in the knowledge graph are initialized. Then, training samples are sampled using a rule-guided training path sample construction method. Next, in the encoder stage, entities and relations are embedded using a knowledge graph embedding model to construct the input token sequence. In the decoder stage, the token sequence from the encoder is decoded autoregressively using a Causal Transformer to generate the next inference sequence. Finally, the co-occurrence patterns of the inference sequence are learned using a teacher forcing parallel training method. Through a sequence-generative inference process that dynamically interacts with the knowledge graph environment, the next action can be generatively inferred based on the current inference state. By using a Decision Transformer architecture to process the inference sequence in parallel, faster knowledge reasoning is achieved, improving inference efficiency and enhancing inference interpretability.

[0057] Furthermore, based on the above method, this embodiment of the invention also provides a sequence-generative knowledge reasoning system based on DecisionTransformer, comprising: a rule mining module, a model building module, and a knowledge reasoning module, wherein,

[0058] The rule mining module is used to extract entities and relationships between entities from the target knowledge graph and to summarize structured rule sequences in the target knowledge graph.

[0059] The model building module is used to build a DecisionTransformer reasoning model for knowledge reasoning and dynamic interaction with the graph environment. When the DecisionTransformer reasoning model interacts dynamically with the graph environment, it uses a rule-guided approach to build training path samples for the reasoning model and uses these training path samples to train the DecisionTransformer reasoning model to learn the co-occurrence ability of rules in the path.

[0060] The knowledge reasoning module is used to perform knowledge reasoning based on the extracted entities, relationships between entities, and rule sequences, using the Decision Transformer reasoning model, and output the reasoning results.

[0061] Unless otherwise specifically stated, the relative steps, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0062] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0063] The units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations are not considered to be beyond the scope of this invention.

[0064] Those skilled in the art will understand that all or part of the steps in the above methods can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Optionally, all or part of the steps in the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiments can be implemented in hardware or as a software functional module. This invention is not limited to any particular combination of hardware and software.

[0065] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A sequence-generative knowledge reasoning method based on Decision Transformer, used for knowledge graph reasoning in information retrieval, characterized in that, Includes the following content: Extract entities and relationships between entities from the target knowledge graph, and summarize structured rule sequences in the target knowledge graph, which is a knowledge graph constructed from information retrieval corpus; A Decision Transformer reasoning model is constructed for knowledge reasoning and dynamic interaction with the graph environment. When the Decision Transformer reasoning model interacts dynamically with the graph environment, it adopts a rule-guided approach to construct training path samples for the reasoning model and uses these training path samples to train the Decision Transformer reasoning model to learn the co-occurrence ability of rules in the path. Based on the extracted entities, relationships between entities, and rule sequences, for a given head entity e s Based on the query q, knowledge reasoning is performed through the Decision Transformer reasoning model to obtain the complete reasoning process sequence τ, and the tail entity e of the process sequence. o This refers to the output reasoning result; The Decision Transformer inference model consists of an encoder and a decoder. In the encoder, each entity and relation is embedded to generate a token sequence, and the order of inference is distinguished by adding positional encoding to the embedded representation of the entity or relation. In the decoder, the next inference action is output by decoding the sequence preceding the current learned sequence item. The decoder of the inference model uses the Causal Transformer model to decode the token sequence of the encoder and generate the next inference sequence; The rule-guided method includes the rule induction method AnyBURL; The Decision Transformer inference model uses cross-entropy as the target loss function and employs label smoothing to smooth the loss function. The target loss function is expressed as: Where τ represents the sequence, k represents the sequence index, V represents the dictionary consisting of the elements in the sequence, and α i The weights represent the smoothness of the labels, q represents the query, and p(i|q,τ) ＜k ) represents the probability distribution of the sequence generated based on the current sequence and the next step of the query, ∈ is a set hyperparameter, and K represents the total number of sequence categories.

2. The sequence-generative knowledge reasoning method based on Decision Transformer according to claim 1, characterized in that, In mining rules from the target knowledge graph, structured rules are summarized in the target knowledge graph using rule induction methods; and rules with confidence scores higher than the preset score threshold are selected using preset score thresholds.

3. The sequence-generative knowledge reasoning method based on Decision Transformer according to claim 1, characterized in that, In the training path samples of the inference model constructed using the rule-guided approach, firstly, all k random walk paths under the preset query conditions are traversed using the random walk method; then, the entity and relation representations in the graph are initialized using the graph embedding method, and each path of all entity and relation element pairs in the random walk is traversed. Then, based on the path rule reward and the semantic similarity between the path and the query conditions, all random walk paths are ranked in importance, and the top N paths are selected as training path samples.

4. The sequence-generative knowledge reasoning method based on Decision Transformer according to claim 3, characterized in that, In ranking the importance of all random walk paths based on path-specific rewards and semantic similarity between the path and query conditions, the formula is used. To calculate path scores, importance is ranked using the score rankings, where p i Let i represent path i, c represent the semantic similarity between the corresponding path and the query condition. i This represents the rule reward for the corresponding path, where λ is the weight parameter.

5. The sequence-generative knowledge reasoning method based on Decision Transformer according to claim 3 or 4, characterized in that, Semantic similarity between path and query conditions (a) i The value is calculated by using path embedding representation and the attention value of preset query conditions. The calculation process is represented as: a i =softmax(q) T r i ),r i =Ap ij , where r i For all embedding representations of the corresponding path samples, p ij Let q represent the entity and relation element pairs corresponding to this path sequence, q represent the query conditions, and A represent the weight matrix composed of the corresponding attention values.

6. The sequence-generative knowledge reasoning method based on Decision Transformer according to claim 3 or 4, characterized in that, The rule reward for a path is obtained through the confidence score of the rule. If the path sequence of the random walk corresponds to the extracted rule, the confidence score of the rule is used as the rule reward. If the random walk path satisfies multiple rules at the same time, the rule with the highest confidence score is selected as the matching rule, and the confidence score of the matching rule is used as the rule reward.

7. A sequence-generative knowledge reasoning system based on Decision Transformer, used for knowledge graph reasoning in information retrieval, characterized in that, It includes: a rule mining module, a model building module, and a knowledge reasoning module, among which, The rule mining module is used to extract entities and relationships between entities from the target knowledge graph and to summarize structured rule sequences in the target knowledge graph, which is a knowledge graph constructed from information retrieval corpus. The model building module is used to build a Decision Transformer reasoning model for knowledge reasoning and dynamic interaction with the graph environment. When the Decision Transformer reasoning model interacts dynamically with the graph environment, it uses a rule-guided approach to build training path samples for the reasoning model and uses these training path samples to train the Decision Transformer reasoning model to learn the co-occurrence ability of rules in the path. The knowledge reasoning module is used to perform reasoning based on the extracted entities, relationships between entities, and rule sequences, for a given head entity e. s Based on the query q, knowledge reasoning is performed through the Decision Transformer reasoning model to obtain the complete reasoning process sequence τ, and the tail entity e of the process sequence. o This refers to the output reasoning result; The Decision Transformer inference model consists of an encoder and a decoder. In the encoder, each entity and relation is embedded to generate a token sequence, and the order of inference is distinguished by adding positional encoding to the embedded representation of the entity or relation. In the decoder, the next inference action is output by decoding the sequence preceding the current learned sequence item. The decoder of the inference model uses the Causal Transformer model to decode the token sequence of the encoder and generate the next inference sequence; The rule-guided method includes the rule induction method AnyBURL; The Decision Transformer inference model uses cross-entropy as the target loss function and employs label smoothing to smooth the loss function. The target loss function is expressed as: Where τ represents the sequence, k represents the sequence index, V represents the dictionary consisting of the elements in the sequence, and α i The weights represent the smoothness of the labels, q represents the query, and p(i|q,τ) ＜k ) represents the probability distribution of the sequence generated based on the current sequence and the next step of the query, ∈ is a set hyperparameter, and K represents the total number of sequence categories.

Images