Strengthened path reasoning algorithm based on script event graph

Abstract

In order to solve the problem of event prediction reasoning, the invention provides an enhanced path reasoning algorithm based on a fused context script event graph. By analyzing the limitation of an existing event prediction reasoning algorithm, an event prediction reasoning problem is modeled as a Markov decision process (MDP), and a reinforcement learning framework including a state space, an action space, a strategy function, a transfer function and a reward mechanism is constructed. In the aspect of algorithm design, design implementation of the whole algorithm is completed through a graph structure learning module, an action scoring module, a random strategy generation module, a reward calculation module and a strategy optimization module. According to the model, training and test verification are carried out on an MCNC and a self-built building construction accident report data set, the result shows that the model is superior to other existing prediction models in a downstream event prediction reasoning task, the accuracy rate is improved by 2.61% compared with an optimal RoBERTa-RF model, the result has statistical significance, and the method is suitable for popularization and application. According to the method, a new methodology framework is provided for an event prediction task, and a new path is opened up for cognitive calculation research of an event atlas through an interpretable path reasoning mechanism.

Description

Technical Field

[0001] The present invention relates to the field of event prediction in natural language processing, and aims to achieve efficient event prediction reasoning in the presence of an existing event knowledge description graph structure. Background Art

[0002] In the field of natural language processing, constructing graphs containing event knowledge based on real-world event descriptions is crucial for improving the performance of downstream tasks such as event prediction and retrieval. Therefore, effectively learning representations for nodes and their relationships within graphs and leveraging graph structures for efficient reasoning has become a hot topic in current research. Currently, predictive reasoning algorithms based on event knowledge graphs can be categorized into three main types: static reasoning based on embedding representations, propagation reasoning based on graph neural networks, and rule-based reasoning based on symbolic logic. Embedding-based static reasoning methods use graph embedding techniques (such as TransE and RGCN) to map event nodes and their relationships into a low-dimensional vector space and then use vector similarity for association prediction. While these methods perform well for simple event association prediction tasks, they struggle to model complex temporal logical relationships. For example, in construction safety management scenarios, if strict dependencies such as "scaffolding erection → acceptance inspection → high-altitude work" are not executed sequentially, static embedding methods cannot effectively capture these temporal constraints, leading to biased risk prediction. Similarly, propagation inference methods based on graph neural networks employ models such as Graph Attention Networks (GAT) and GraphSAGE to achieve local structural inference by aggregating neighbor nodes. While these models can capture explicit connections between nodes, they often suffer from information decay when dealing with long, multi-hop causal chains, resulting in reduced inference performance. Finally, rule-based inference methods based on symbolic logic combine knowledge representation with logical rules, improving model interpretability through manually defined rules. However, their rule generalization capabilities are limited, requiring frequent adjustments to the rule base in dynamic scenarios, making adaptive inference difficult.

[0003] To address the limitations of existing methods, the present invention constructs a reinforcement path inference algorithm based on a script event graph. This model models the event inference process as a Markov decision process (MDP). Its operating mechanism is as follows: after the agent receives the initial question query, it implements iterative search and optimization strategy generation of the inference path based on the reinforcement learning architecture, and finally outputs the optimal action sequence. From the perspective of the algorithm framework, reinforcement learning is essentially a machine learning paradigm that achieves strategy optimization through multiple rounds of interaction between the agent and the environment. Its mathematical representation includes five core elements: the state space S represents the environmental information in which the system is located, the action space A defines the set of optional operations, the policy function π(a|s) determines the mapping rule from state to action, the state transition model P(s'|s,a) describes the evolution of the environment after the action is executed, and the reward function R(s,a,s') quantifies the immediate utility of the decision-making behavior. These five elements constitute the theoretical cornerstone of reinforcement learning, and a rigorous formal definition of them will be developed later.

[0004] (1) State space S

[0005] S={s0,s1,s2,...,s n} represents the state space, the state s reached at step n n =((n i ),(n q ,r q ))∈S consists of two key parts: local state information and global state information. The local state information refers to the event node that the agent finally reaches through random walks, reflecting the timely progress of the reasoning process. The global state information is composed of the event nodes and their associations in the query tuple, which serves as the context information shared by all states.

[0006] (2) Action space A

[0007] A={A0,A1,A2,...,A n} represents the action space, which represents the set of decisions that the agent can make in a given state. where a n-1,k ={(r n,k ,ε n,k )}. The agent’s behavior selection space is dynamically generated by the topological structure of its state. Specifically, when the system is in state s n =((ε n-1 ),(ε q ,r q )) (where ε n-1 Represents the currently activated event node), the generation of the executable action set follows the spatial expansion principle driven by the graph structure, by extracting the event node ε n-1 The topological connection relationship of each of them is used to obtain its direct successor edge set. and its target node set

[0008] Construct the basic action space A base In order to realize the state maintenance mechanism, a special action is introduced self =(r null ,ε n-1 ) to achieve decision suspension, so that the agent can make the decision within the preset number of iterations T max Under the premise of maintaining the current state and accumulating information until the path termination condition is met. The final effective action space representation is A n =A base ∪{a self}.

[0009] (3) Strategy π(a|s)

[0010] This chapter uses random strategies as reinforcement learning modeling. The decision-making behavior of the agent is determined by the parameterized conditional probability distribution π θ :S×A→[0,1] representation, where Represents a trainable policy parameter vector. Specifically, for any state-action pair (s, a)∈S×A, the policy function π θ (a|s) defines the conditional probability density of selecting action a under the observation state s, satisfying the normalization constraint ∫ A π θ (a|s)da=1.

[0011] (4) State transition model P(s'|s,a)

[0012] The state transition function P(s'|s,a) is the core component of the Markov decision process. It quantifies the probability density of the system transitioning to a new state s'∈S after the agent takes action a∈A in a specific state s∈S. This function satisfies the basic properties of probability distribution, namely the normalization constraint ∑ s'∈S P(s'|s,a)=1. This property ensures the probabilistic completeness of the state transition process and provides a theoretical basis for policy evaluation and optimization in reinforcement learning algorithms. In practice, this function is implicitly learned through the interaction data between the agent and the environment.

[0013] (5) Reward function R(s,a,s')

[0014] In the script event graph-based reinforced path reasoning algorithm SERP-RL, the cumulative reward function represents the temporal aggregation effect of the reward signal during the agent's path exploration process, where γ∈[0,1) is the discount factor, Represents a single reasoning path trajectory. When the agent is at the maximum exploration step T maxArrival at target event node ε target When you get the unit reward r immediately T =1, indicating a deterministic reward for task completion. T ≠ε target , then the reward function comprehensive path score and topological correlation measure C topo (ε T ,ε target ), calculated as r T =αS path +(1-α)C topo , where φ(·) is the edge weight function and α∈[0,1] is the balance coefficient. Summary of the Invention

[0015] To address the limitations of existing methods, this paper constructs an enhanced path reasoning algorithm based on a script event graph. This algorithm models the event reasoning task as a Markov decision process (MDP). The technical solution used to deconstruct complex event relationships from a path analysis perspective is as follows:

[0016] Step 1: This paper proposes an enhanced path reasoning algorithm based on the script event graph for event prediction and reasoning tasks in natural language processing. After constructing an event representation architecture for event description text, the graph structure must first be feature encoded. The system uses graph structure representation learning to extract the semantic embedding features of vertices in the script event graph, the topological relationship features of associated edges, and the contextual features of multiple paths;

[0017] Step 2: Action scoring process. Based on the constructed vertex representation Relational Representation q And the path encoding p l , perform joint special fusion, model the semantic features of target relations and target vertices respectively through the multi-layer perceptron (MLP) framework based on the Markov decision process, construct the state-action value function, and generate quantitative evaluation indicators for candidate actions.

[0018] Step 3: Generate a random walk policy. According to the mathematical definition of the reinforcement learning task, when given the current state, select the best action from the candidate action set.

[0019] Step 4: Calculate policy rewards. In a reinforcement learning framework, the design of the reward function must be strictly aligned with the algorithm's optimization objectives. The reinforcement path inference algorithm proposed in this paper optimizes the agent's long-term cumulative reward, rather than pursuing the immediate benefits of a single step. A reward calculation mechanism is established that combines soft guiding rewards with hard constraint rewards.

[0020] Step 5: Strategy optimization. For a given query tuple (εq ,r q ,? ,c q ), where ? is a part of the query tuple, representing the unknown event to be inferred. The agent starts with the initial node n q =(ε q ,c q ) as the starting point, a multi-step random search is performed until the terminal state is reached and an inference path is generated. The system then calculates the comprehensive score of the path based on the soft and hard mixed reward mechanism in step 4 and updates the parameters of the policy learning module using the policy gradient method. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is: Flowchart of the enhanced path reasoning model algorithm.

[0022] Figure 2 Figure 1: Event prediction experimental results of the reinforced path reasoning algorithm based on the script event graph. DETAILED DESCRIPTION

[0023] Given that the temporal characteristics exhibited by script event graphs during event reasoning bear significant resemblance to those of path sequence decision problems, this property provides a theoretical basis for transforming event reasoning problems into reinforcement learning problems. Specifically, the two share common modeling features in terms of state transitions, action selection, and temporal decision making. This enables reinforcement learning-based solutions to effectively capture the temporal dependencies in event reasoning. Therefore, this paper employs a reinforcement learning framework to model this type of problem, leveraging its advantages in sequential decision making. Specifically, the context-integrated script event graph is treated as the internal data structure of the agent. This paper models the reasoning process using a Markov decision process (MDP). In this framework, the agent receives a question query as input, and its behavioral decisions are governed by a policy learning module. Based on the reinforcement learning algorithm architecture, the agent generates multiple exploration paths by executing random walks. This process continues until a preset termination condition is met. After multiple iterations, the system collects all generated path samples. For each path, the study calculates the corresponding soft-guided reward and hard-constrained reward indicators, and derives the cumulative reward value based on these indicators. To optimize model performance, the present invention designs an objective function aimed at maximizing cumulative reward and employs a policy gradient algorithm to iteratively update the parameters of the policy learning module. After parameter optimization, the system uses the improved policy learning module to process queries and input the final inference conclusion. The resulting inference path not only provides the final inference result but also serves as supporting evidence for the event deduction process, significantly improving the interpretability of the entire inference system.

[0024] Specific methods

[0025] Aiming at the problem of event reasoning, this paper proposes a reinforced path reasoning algorithm based on script event graph. The implementation process of the algorithm includes graph structure learning, action reward scoring, random strategy generation and strategy optimization.

[0026] Figure 1 It is a flowchart of the algorithm for the enhanced path reasoning model. The algorithm framework proposed in the present invention adopts a multi-stage collaborative optimization mechanism, and its workflow can be divided into the following key steps: First, in the graph structure feature encoding stage, the system extracts the semantic embedding features of the vertices in the script event graph, the topological relationship features of the associated edges, and the contextual features of multiple paths through graph structure representation learning; then the action scoring module constructs a state-action value function based on the Markov decision process to generate quantitative evaluation indicators for candidate actions; the obtained evaluation parameters are processed by the random strategy generation module to generate a probabilistic action decision distribution, which is used to know the exploration direction of the intelligent agent. After the intelligent agent completes the autonomous exploration of the preset step length, it generates a differentiated trajectory set through multiple iterations, and then comprehensively calculates the soft reward and hard constraint indicators to construct a reward function. The algorithm goal is transformed into solving the optimization problem of maximizing the cumulative benefits of multiple rounds of exploration.

[0027] At the parameter optimization level, this study adopts a reinforcement learning paradigm based on a policy gradient algorithm. The policy optimizer continuously adjusts the model's weight parameters through a gradient ascent mechanism, so that the expected reward function converges to the optimal solution. Specifically, after the system obtains multiple reasoning trajectories through Monte Carlo sampling, it first calculates the comprehensive benefit indicator at the trajectory level, and then backpropagates the policy gradient to update the parameter matrix of the policy network. After sufficient training to obtain the optimal parameter configuration, the reasoning model can not only output the final reasoning conclusion when responding to query questions, but also generate a complete chain of evidence including the intermediate decision-making process. This explicit reasoning trajectory not only meets the technical specifications of explainable artificial intelligence, but also provides a logical verification basis for complex event reasoning tasks, forming an explainable logical chain.

[0028] The graph structure learning is responsible for mapping the original script event graph structure and query tuple into a computable distributed representation space, capturing the event node embedding representation, relationship embedding representation and path embedding representation, etc. First, the relationship between nodes and events in the script event graph is embedded. For the event relationship r i Assign a random initialization vector For graph nodes c i =(t i ,l i ), according to the event ε in Chapter 3 iThe core components of the event are encoded and then concatenated to form the event node embedding representation. Specifically, the event participant P, space L, time T and event semantics S are represented by vectors respectively, where the event participant P belongs to the static entity embedding Time T is vectorized using the time encoding function Ψ(.), as shown in formula (1). The encoding process of space L and event semantics S is obtained as described in Chapter 3. and Finally, the embedding vectors of each event component are concatenated to obtain the embedding representation of the event node, as shown in formula (2).

[0029]

[0030] In formula (1), the parameter d corresponds to the total number of points of the time kernel approximate sampling, where the characteristic frequency vectors ω1, ω2, ω3, ..., ω d It is composed of an arithmetic progression of equally spaced numbers in the interval [0, d]. This design ensures that the mathematical properties of the time encoding function Ψ(.) satisfy the basic property of translation invariance, that is, the statistical characteristics of the time series remain constant under different time offsets.

[0031] At the completion event node n i After modeling the embedded representation of the event, it is necessary to further generate vector representations of invisible events in the script event graph. Traditional methods are mostly based on the graph neural network framework, which realizes the representation learning of implicit events through information aggregation of domain nodes. This chapter introduces the semantic consistency hypothesis: if event instances share the same event center word K and semantic logical event relationship r, then their representations should be similar. Based on this, the algorithm first calculates the common features of similar events through formula (3) Among them E K,Rs Indicates that the central word K and the semantic logical event relationship R are satisfied at the same time s The event instance set of |E K,Rs | indicates its cardinality.

[0032]

[0033] Furthermore, the graph structure learning module is based on the extracted common features Combined with dynamic context information, the invisible events are iteratively represented and updated. As shown in formula (4-4), the vector representation of the current moment is From the previous state It is generated by weighted fusion with domain features, where λ∈(0,1) is the smoothing coefficient, KR s (ε q ) represents the target event ε q The set of associated center word-relation pairs, |KR s (εq )| is its scale parameter.

[0034] After completing the vector representation learning of vertices and relations in the script event graph, the algorithm further constructs the semantic embedding model of the reasoning path. As shown in formula (5), the reasoning path p l It consists of an alternating sequence of vertices and relations, where the starting node The initial state of the corresponding query relationship, the terminal node n l =(ε l ,c l ) represents the target event explored in step l, and r0 is the preset initial semantic association.

[0035]

[0036] To capture the dynamic evolution characteristics of the path, this framework uses the gated recurrent unit (GRU) architecture to model the inference path sequence. Formula (6) defines the recursive process of path encoding: the encoding state p of the current step l The encoding result p from the previous step l-1 The relationship r with the current step l-1 , vertex representation In particular, the initial code p0 is generated by the zero input state and the initial path information The fusion calculation is obtained.

[0037] If a self-loop operation is triggered during the path exploration process, the current encoding state is kept unchanged to avoid redundant updates.

[0038]

[0039] Based on the constructed vertex representation in the script event graph Relation Embedding q and path encoding p l , the algorithm uses the multi-layer perceptron (MLP) framework to model the semantic features of the target relationship and the target vertex respectively. As shown in formula (7), the target relationship representation It is generated by the path-vertex-relationship joint feature through linear transformation and nonlinear activation function, where W r and W l is the trainable weight matrix.

[0040]

[0041] Similarly, the target vertex representation By mapping to high-dimensional space through a similar structure (Formula 8), W e is the corresponding parameter matrix.

[0042]

[0043] To quantify the semantic relevance of action selection, the system calculates the target relationship Relationship with candidate r l The cosine similarity of (Formula 4-9), and the target vertex With candidate vertices The similarity score of (Formula 10).

[0044]

[0045] Final action score φ(a l ,s l ) is obtained by weighted fusion of the above two types of scores, as shown in formula (11), where the weight coefficient β n Dynamically adjusted by the attention mechanism, as shown in Equation (12). This mechanism integrates the contextual features of paths, vertices, and relationships through the Sigmoid function, enabling the agent to adaptively focus on the core components of events or key relationship types during random walks. In particular, when the reasoning path lacks explicit logical rules, the model significantly improves the robustness of path search by strengthening the implicit association between query tuples and target events / relationships.

[0046]

[0047]

[0048] The algorithm is given the current state s l When, from the candidate action set Specifically, the system first calculates the utility score φ(a l,k ,s l ), and then the Softmax function is applied to normalize the scores, thereby generating the state-dependent policy distribution function π(a l,k |s l ), as shown in formula (13). This function maps the action selection probability to the exponentially normalized score ratio, ensuring that the policy output satisfies the probability space constraints while retaining the decision preference for high-scoring actions.

[0049]

[0050] In the reinforcement learning framework, the design of the reward function must be strictly aligned with the algorithm optimization goal. The reinforcement path inference model proposed in this paper optimizes the agent's long-term cumulative reward rather than pursuing the immediate benefits of a single step. The specific reward mechanism is designed as follows: If the agent finally reaches the target event node, that is, ε L =ε target , then the unit is given a reward R(sL )=1, otherwise the reward value is 0, as shown in formula (14). This design corresponds to the "hit reward", which directly encourages the agent to explore the target node and can be regarded as a hard constraint reward. In order to avoid relying solely on hard constraint rewards, which leads to the neglect of samples with the correct path direction but not reaching the target, the model introduces a reward mechanism that mixes soft guidance rewards and hard constraint rewards. As shown in formula (15), when the agent terminates at a non-target node, the total reward is calculated by the soft reward function f(g), which is a weighted fusion of the event quadruple score reward p(true|g) and the path relationship similarity score. It is realized as shown in formula (16), where η∈[0,1] is the balance coefficient.

[0051]

[0052] The event quadruple scoring is implemented based on the TransE model, which transforms the inference path p l With query node Mapping into path relationship representation As shown in formula (17), the path score reward is given by Relationship with query r q As shown in formula (18), this design makes the reward value higher when the path is closer to the query relationship semantics, thereby guiding the agent to explore logically coherent reasoning paths.

[0053]

[0054] For a given query tuple (ε q ,r q ,? ,c q ), where ? is a part of the query tuple, representing the unknown event to be inferred. The agent starts with the initial node n q =(ε q ,c q ) as the starting point, a multi-step random search is performed until the terminal state is reached and an inference path is generated. The system then calculates the path's comprehensive score based on the soft and hard hybrid reward mechanism (Formulas 14 and 18) and updates the parameters of the policy learning module using the policy gradient method.

[0055] Specifically, the optimization goal of the algorithm is to maximize the training set F train The expected cumulative reward of all samples in is as shown in formula (19), where the outer expectation represents the integral of the distribution of training samples and the inner expectation represents the integral of the strategy π θ Generates statistical average of action sequences. Total reward It is composed of the weighted combination of the hard hit bonus and the soft guidance score, as shown above.

[0056]

[0057] During the parameter update process, the policy gradient is approximated by Monte Carlo sampling: all event quads in the training set are traversed, and the product of the logarithmic probability of each action and the corresponding reward value is accumulated, as shown in formula (20). Ultimately, the policy learning module, after sufficient iterative optimization, can reason about the query problem based on the optimal parameter configuration, outputting both a high-confidence reasoning conclusion and a traceable decision path.

[0058]

[0059] To verify the effectiveness of the algorithm, we selected the Multiple Choice Narrative Cloze (MCNC) evaluation paradigm and a self-built accident report dataset as validation datasets. The self-built accident report dataset mainly sources 1,025 reports selected from accident investigation reports issued by official agencies such as the Ministry of Housing and Urban-Rural Development of China, the Ministry of Emergency Management, and the Housing and Urban-Rural Development Departments of various provinces and cities. Sentence segmentation, format conversion, and effective content extraction (i.e., deleting irrelevant information) were performed on the self-built accident report dataset, resulting in 57,454 sentences.

[0060] Figure 2This is the event prediction experimental result diagram of the enhanced path reasoning algorithm based on the script event graph proposed in this invention. The algorithms involved in the comparison are Event-Comp model, PairLSTM network, SGNN neural network model based on NEG structure, SAM-NET model, GraphTransformer model, and RoBERTa-RF pre-trained language model. In order to verify whether the improvement of model performance in downstream tasks brought about by the change of event representation architecture is statistically significant, and to prove that the performance improvement is not caused by random factors, that is, to enhance the credibility of the experimental results, the T-test statistical hypothesis testing method is introduced, and the accuracy (number of correctly predicted events / total number of events in the test set) is used as the evaluation index. The experimental results of the comparative experiment with the above models are shown in Table 4-2. The p value obtained by the T test is less than 0.01. According to the experimental results in the table, on the MCNC dataset, the SERP-RL model proposed in this chapter significantly outperforms other comparison models (such as RoBERTa-RF's 58.66% and GraphTransformer's 58.01%) with an accuracy of 61.37%, demonstrating the effectiveness of the algorithm in this chapter. This advantage is due to its integration of reinforcement learning's event relationship constraint strategy, which can effectively model complex spatiotemporal dependencies. Traditional methods (such as EventComp's 49.57%) and basic sequence models (such as PairLSTM's 50.83%) performed the worst due to their lack of deep reasoning ability about event associations. T-test verification (p<0.01) shows that the performance differences between the models are statistically significant, indicating that the superiority of the algorithm model proposed in this paper in the event prediction task is not a random fluctuation.

[0061] On a self-built dataset of construction accident reports, the proposed enhanced path reasoning algorithm based on scripted event graphs continued to perform well. In terms of event prediction evaluation metrics, its accuracy outperformed most of the compared algorithms and was 1.19% higher than the powerful pre-trained model method RoBERTa-RF. The results in the table show that each algorithm performed better on the accident report dataset than on the MCNC dataset. This is because the accident report dataset contains more event information, including time and space, and the event descriptions are more detailed, resulting in a more complete event representation structure. This also proves that when performing event prediction reasoning, if the model can learn more relevant event knowledge, it will help to obtain more accurate prediction results.

Claims

1. The present invention aims to design an enhanced path reasoning algorithm based on a script event graph, characterized in that: The technical solutions adopted are: Step 1: This paper proposes an enhanced path reasoning algorithm based on the script event graph for event prediction and reasoning tasks in natural language processing. After constructing an event representation architecture for event description text, the graph structure must first be feature encoded. The system uses graph structure representation learning to extract the semantic embedding features of vertices in the script event graph, the topological relationship features of associated edges, and the contextual features of multiple paths; Step 2: Action scoring process. Based on the constructed vertex representation Relational Representation q And the path encoding p l , perform joint special fusion, model the semantic features of target relations and target vertices respectively through the multi-layer perceptron (MLP) framework based on the Markov decision process, construct the state-action value function, and generate quantitative evaluation indicators for candidate actions. Step 3: Generate a random walk policy. According to the mathematical definition of the reinforcement learning task, when given the current state, select the best action from the candidate action set. Step 4: Calculate policy rewards. In a reinforcement learning framework, the design of the reward function must be strictly aligned with the algorithm's optimization objectives. The reinforcement path inference algorithm proposed in this paper optimizes the agent's long-term cumulative reward, rather than pursuing the immediate benefits of a single step. A reward calculation mechanism is established that combines soft guiding rewards with hard constraint rewards. Step 5: Strategy optimization. For a given query tuple (ε q ,r q ,? ,c q ), where ? is a part of the query tuple, representing the unknown event to be inferred. The agent starts with the initial node n q =(ε q ,c q ) as the starting point, a multi-step random search is performed until the terminal state is reached and an inference path is generated. The system then calculates the comprehensive score of the path based on the soft and hard mixed reward mechanism in step 4 and updates the parameters of the policy learning module using the policy gradient method.

2. As claimed in claim 1, it is characterized in that The vertex representation constructed in step 2 based on the script event graph Relation Embedding q and path encoding p l The algorithm uses a multi-layer perceptron (MLP) framework to model the semantic features of target relations and target vertices, where the target relation representation The target vertex representation is generated by the path-vertex-relationship joint feature through linear transformation and nonlinear activation function. By mapping similar structures to high-dimensional space, the system then calculates the target relationship to quantify the semantic relevance of action selection. Relationship with candidate r l The cosine similarity of , and the target vertex With candidate vertices The final action score φ(a l ,s l ) is obtained by weighted fusion of the two aforementioned scoring methods. This mechanism integrates contextual features of paths, vertices, and relationships through a sigmoid function, enabling the agent to adaptively focus on core event components or key relationship types during random walks. Especially when the inference path lacks explicit logical rules, the model significantly improves the robustness of path search by strengthening the implicit association between query tuples and target events / relationships.

3. As claimed in claim 1, it is characterized in that In step 3, the system first calculates the utility score φ(a l,k ,s l ), and then the Softmax function is applied to normalize the scores, thereby generating the state-dependent policy distribution function π(a l,k |s l ), which maps the action selection probability to the exponentially normalized score ratio, ensuring that the policy output satisfies the probability space constraints while retaining the decision preference for high-scoring actions.

4. The method according to claim 1, wherein: The specific reward mechanism in step 4 is designed as follows: If the agent finally reaches the target event node, that is, ε L =ε target , then the unit is given a reward R(s L )=1, otherwise the reward value is 0. This design corresponds to a "hit reward" that directly encourages the agent to explore the target node and can be regarded as a hard constraint reward. To avoid relying solely on hard constraint rewards, which may lead to the neglect of samples with correct path directions but not reaching the target, the model introduces a reward mechanism that combines soft guidance rewards and hard constraint rewards. When the agent terminates at a non-target node, the total reward is calculated by the soft reward function f(g), which is a weighted fusion of the event quadruple score reward p(true|g) and the path relationship similarity score. Then, the event quadruple score is realized based on the TransE model, and the inference path p is transformed through the nonlinear transformation layer. l With query node Mapping into path relationship representation Path scoring rewards are given by Relationship with query r q This design makes the reward value higher the closer the path is to the query relationship semantics, thereby guiding the agent to explore logically coherent reasoning paths.

5. The method according to claim 1, wherein: Step 5 The optimization goal is to maximize the training set F train The expected cumulative reward of all samples in , where the outer expectation represents the integral of the distribution of training samples, and the inner expectation represents the integral of the strategy π θ Statistical average of generated action sequences, total reward Composed of a weighted hard hit reward and a soft guidance score, the policy gradient is approximated during parameter updates using Monte Carlo sampling: traversing all event quads in the training set and accumulating the product of the log-probability of each action and the corresponding reward. Ultimately, the policy learning module, after thorough iterative optimization, can reason about the query based on the optimal parameter configuration, outputting both a high-confidence conclusion and a traceable decision path.

Images